二进制安装k8s集群-部署flannel(十一)

Author： kilig
发布时间：May 6, 2022
3168 views
No comments
11226 words
Categories： devops

Flannel是由go语言开发，是一种基于 Overlay 网络的跨主机容器网络解决方案，也就是将TCP数据包封装在另一种网络包里面进行路由转发和通信，Flannel 是 CoreOS 开发，专门用于docker多主机互联的一个工具，简单来说，它的功能是让集群中的不同节点主机创建的容器都具有全局唯一的虚拟IP地址。

在默认的Docker配置中，每个节点上的Docker服务会分别独立负责自己所在节点容器的IP地址分配，这样就导致一个问题，不同节点上容器获得的IP地址可能一样，容器之间无法直接通信，需要借助nat桥接的方式来进行通信。

安装过程比较简单，下载解压，建立软连接！

[root@hdss7-21 ~]# cd /data/soft/  && mkdir -p /usr/local/flannel-v0.11.0

[root@hdss7-21 soft]# wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
[root@hdss7-21 soft]# tar xf flannel-v0.11.0-linux-amd64.tar.gz -C /usr/local/flannel-v0.11.0
[root@hdss7-21 ~]#cd /usr/loacal
[root@hdss7-21 local]# ln -sf flannel-v0.11.0/ flannel
[root@hdss7-21 local]# ll
total 0
lrwxrwxrwx  1 root root  25 Apr 20 14:46 etcd -> etcd-v3.1.20-linux-amd64/
drwxr-xr-x  4 etcd etcd 166 Apr 20 15:50 etcd-v3.1.20-linux-amd64
lrwxrwxrwx  1 root root  16 Apr 26 17:33 flannel -> flannel-v0.11.0/
drwxr-xr-x  2 root root  64 Apr 26 17:33 flannel-v0.11.0
lrwxrwxrwx  1 root root  18 Apr 21 11:30 kubernetes -> kubernetes-v1.15.2
drwxr-xr-x  4 root root  50 Apr 21 11:32 kubernetes-v1.15.2

拷贝证书，需要与etcd通信并配置，所以需要拷贝以下证书

[root@hdss7-21 flannel]# mkdir cert
[root@hdss7-21 cert]#scp hdss7-200:/opt/certs/ca.pem .
[root@hdss7-21 cert]#scp hdss7-200:/opt/certs/client.pem .
[root@hdss7-21 cert]#scp hdss7-200:/opt/certs/client-key.pem .

[root@hdss7-21 cert]#cd ../
[root@hdss7-21 flannel]# vim subnet.env
FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.21.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

[root@hdss7-21 flannel]# vim flanneld.sh
#!/bin/sh
./flanneld \
  --public-ip=10.4.7.21 \
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 \
  --etcd-keyfile=./cert/client-key.pem \
  --etcd-certfile=./cert/client.pem \
  --etcd-cafile=./cert/ca.pem \
  --iface=ens33 \
  --subnet-file=./subnet.env \
  --healthz-port=2401

[root@hdss7-21 flannel]# chmod +x flanneld.sh
[root@hdss7-21 flannel]# mkdir -p /data/logs/flanneld


[root@hdss7-21 flannel]# vim /etc/supervisord.d/flannel.ini
[program:flanneld-7-21]
command=/usr/local/flannel/flanneld.sh                       ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/usr/local/flannel                                ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

设置flannel的网络插件模式为host-gw

[root@hdss7-21 etcd]# ./etcdctl set /coreos.com/network/config '{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}'
{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}

查看
[root@hdss7-21 etcd]# ./etcdctl get /coreos.com/network/config
{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}

其他节点基本和hdss7-21相同，注意修改一下文件：

[root@hdss7-22 flannel]# vim subnet.env
FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.22.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

flanneld.sh
#!/bin/sh
./flanneld \
  --public-ip=10.4.7.22 \
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 \
  --etcd-keyfile=./cert/client-key.pem \
  --etcd-certfile=./cert/client.pem \
  --etcd-cafile=./cert/ca.pem \
  --iface=ens33 \
  --subnet-file=./subnet.env \
  --healthz-port=2401

chmod +x flanneld.sh

vim /etc/supervisord.d/flannel.ini
[program:flanneld-7-22]
command=/usr/local/flannel/flanneld.sh                       ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/usr/local/flannel                                ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

[root@hdss7-22 flannel]# mkdir -p /data/logs/flanneld/

其他方式（没有实验过VxLAN模式）。

使用方法：

1、先停止flennel.sh ---通过supervisor stop flanneld-7-[21.22]
2、删除host-gw模型创建的路由
route del -net 172.7.21.0/24 gw 10.4.7.21     hdss7-22上
route del -net 172.7.22.0/24 gw 10.4.7.22     hdss7-21上
3、在etcd节点修改
./etcdctl get /coreos.com/network/config
./etcdctl rm /coreos.com/network/config
etcd]# ./etcdctl set /coreos.com/network/config '{"Network": "172.7.0.0/16", "Backend": {"Type": "VxLAN"}}'
4、supervisorctl start flanneld-7-21
    supervisorctl start flanneld-7-22
5、查看ifconfig 会多了一个flannel 1的设备，route -n是没有路由的


2.1.17.flannel直接路由模型（智能判定）
类似与mysql日志的mixed模式

./etcdctl set /coreos.com/network/config '{"Network": "172.7.0.0/16", "Backend": {"Type": "VxLAN","Directrouting": true}}'


[root@hdss7-21 script]# iptables-save |grep -i postrouting
:POSTROUTING ACCEPT [68:4092]
:KUBE-POSTROUTING - [0:0]
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A POSTROUTING -s 172.7.21.0/24 ! -o docker0 -j MASQUERADE
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE


flennel优化
yum install iptables-services  -y
[root@hdss7-21 ~]# systemctl start iptables
[root@hdss7-21 ~]# systemctl enable  iptables

[root@hdss7-21 ~]# iptables -t nat -D POSTROUTING -s 172.7.21.0/24 ! -o docker0 -j MASQUERADE
[root@hdss7-21 ~]# iptables -t nat -I POSTROUTING -s 172.7.21.0/24 ! -d 172.7.0.0/16 ! -o docker0 -j MASQUERADE
[root@hdss7-21 ~]# iptables-save |grep -i postrouting

 iptables -t filter -D INPUT -j REJECT --reject-with icmp-host-prohibited
 iptables -t filter -D FORWARD -j REJECT --reject-with icmp-host-prohibited
##########规则定义#########
10.4.7.21主机上的，来源是172.7.21.0/24段的docker的ip，目标ip不是172.7.0.0/16段，网络发包不从docker0桥设备出站的，才进行SNAT转换

~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]

各自访问对方节点，并查看nginx-access日志，可看到现在暴露的都是容器ip
[root@hdss7-21 ~]#  kubectl  logs -f nginx-ds-jtn62
172.7.22.2 - - [23/Nov/2019:17:46:48 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.38.0" "-"
[root@hdss7-22 ~]# kubectl  logs -f nginx-ds-d5kl8
10.4.7.21 - - [23/Nov/2019:17:01:37 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.38.0" "-"

172.7.21.2 - - [23/Nov/2019:17:43:34 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.38.0" "-"


[root@hdss7-21 ~}#route add -net 172.7.22.0/24 gw 10.4.7.22 dev eth0
[root@hdss7-22~}#route add -net 172.7.21.0/24 gw 10.4.7.21 dev eth0
[root@hdss7-21 flannel]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.7.254      0.0.0.0         UG    100    0        0 eth0
10.4.7.0        0.0.0.0         255.255.255.0   U     100    0        0 eth0
172.7.21.0      0.0.0.0         255.255.255.0   U     0      0        0 docker0
172.7.22.0      10.4.7.22       255.255.255.0   UG    0      0        0 eth0

[root@hdss7-22 flannel]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.7.254      0.0.0.0         UG    100    0        0 eth0
10.4.7.0        0.0.0.0         255.255.255.0   U     100    0        0 eth0
172.7.21.0      10.4.7.21       255.255.255.0   UG    0      0        0 eth0
172.7.22.0      0.0.0.0         255.255.255.0   U     0      0        0 docker0

注意还要优化一条iptables规则：
~]#  iptables -t filter -I FORWARD -d 172.7.21.0/24 -j ACCEPT



flennel优化
yum install iptables-services  -y
[root@hdss7-22 ~]# systemctl start iptables
[root@hdss7-22 ~]# systemctl enable  iptables

[root@hdss7-22 ~]# iptables -t nat -D POSTROUTING -s 172.7.22.0/24 ! -o docker0 -j MASQUERADE
[root@hdss7-22 ~]# iptables -t nat -I POSTROUTING -s 172.7.22.0/24 ! -d 172.7.0.0/16 ! -o docker0 -j MASQUERADE
[root@hdss7-22 ~]# iptables-save |grep -i postrouting

 iptables -t filter -D INPUT -j REJECT --reject-with icmp-host-prohibited
 iptables -t filter -D FORWARD -j REJECT --reject-with icmp-host-prohibited
##########规则定义#########
10.4.7.22主机上的，来源是172.7.22.0/24段的docker的ip，目标ip不是172.7.0.0/16段，网络发包不从docker0桥设备出站的，才进行SNAT转换

~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]

各自访问对方节点，并查看nginx-access日志，可看到现在暴露的都是容器ip
这里的需要注意防火墙的放通问题。

Last modification：May 6, 2022

如果觉得我的文章对你有用，请随意赞赏

二进制安装k8s集群-部署flannel(十一)

kilig • 2022 年 05 月 06 日

<p>Flannel是由go语言开发，是一种基于 Overlay 网络的跨主机容器网络解决方案，也就是将TCP数据包封装在另一种网络包里面进行路由转发和通信，Flannel 是 CoreOS 开发，专门用于docker多主机互联的一个工具，简单来说，它的功能是让集群中的不同节点主机创建的容器都具有全局唯一的虚拟IP地址。</p><p>在默认的Docker配置中，每个节点上的Docker服务会分别独立负责自己所在节点容器的IP地址分配，这样就导致一个问题，不同节点上容器获得的IP地址可能一样，容器之间无法直接通信，需要借助nat桥接的方式来进行通信。</p><p>安装过程比较简单，下载解压，建立软连接！</p><pre><code class="lang-Bash">[root@hdss7-21 ~]# cd /data/soft/  &amp;&amp; mkdir -p /usr/local/flannel-v0.11.0

[root@hdss7-21 soft]# wget https://github.com/coreos/flannel/releases/download/v0.11.0/flannel-v0.11.0-linux-amd64.tar.gz
[root@hdss7-21 soft]# tar xf flannel-v0.11.0-linux-amd64.tar.gz -C /usr/local/flannel-v0.11.0
[root@hdss7-21 ~]#cd /usr/loacal
[root@hdss7-21 local]# ln -sf flannel-v0.11.0/ flannel
[root@hdss7-21 local]# ll
total 0
lrwxrwxrwx  1 root root  25 Apr 20 14:46 etcd -&gt; etcd-v3.1.20-linux-amd64/
drwxr-xr-x  4 etcd etcd 166 Apr 20 15:50 etcd-v3.1.20-linux-amd64
lrwxrwxrwx  1 root root  16 Apr 26 17:33 flannel -&gt; flannel-v0.11.0/
drwxr-xr-x  2 root root  64 Apr 26 17:33 flannel-v0.11.0
lrwxrwxrwx  1 root root  18 Apr 21 11:30 kubernetes -&gt; kubernetes-v1.15.2
drwxr-xr-x  4 root root  50 Apr 21 11:32 kubernetes-v1.15.2</code></pre><h1>拷贝证书，需要与etcd通信并配置，所以需要拷贝以下证书</h1><pre><code class="lang-Bash">[root@hdss7-21 flannel]# mkdir cert
[root@hdss7-21 cert]#scp hdss7-200:/opt/certs/ca.pem .
[root@hdss7-21 cert]#scp hdss7-200:/opt/certs/client.pem .
[root@hdss7-21 cert]#scp hdss7-200:/opt/certs/client-key.pem .

[root@hdss7-21 cert]#cd ../
[root@hdss7-21 flannel]# vim subnet.env
FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.21.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

[root@hdss7-21 flannel]# vim flanneld.sh
#!/bin/sh
./flanneld \
  --public-ip=10.4.7.21 \
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 \
  --etcd-keyfile=./cert/client-key.pem \
  --etcd-certfile=./cert/client.pem \
  --etcd-cafile=./cert/ca.pem \
  --iface=ens33 \
  --subnet-file=./subnet.env \
  --healthz-port=2401

[root@hdss7-21 flannel]# chmod +x flanneld.sh
[root@hdss7-21 flannel]# mkdir -p /data/logs/flanneld

[root@hdss7-21 flannel]# vim /etc/supervisord.d/flannel.ini
[program:flanneld-7-21]
command=/usr/local/flannel/flanneld.sh                       ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/usr/local/flannel                                ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)
</code></pre><h1>设置flannel的网络插件模式为host-gw</h1><pre><code class="lang-Bash">[root@hdss7-21 etcd]# ./etcdctl set /coreos.com/network/config '{&quot;Network&quot;: &quot;172.7.0.0/16&quot;, &quot;Backend&quot;: {&quot;Type&quot;: &quot;host-gw&quot;}}'
{&quot;Network&quot;: &quot;172.7.0.0/16&quot;, &quot;Backend&quot;: {&quot;Type&quot;: &quot;host-gw&quot;}}

查看
[root@hdss7-21 etcd]# ./etcdctl get /coreos.com/network/config
{&quot;Network&quot;: &quot;172.7.0.0/16&quot;, &quot;Backend&quot;: {&quot;Type&quot;: &quot;host-gw&quot;}}</code></pre><p>其他节点基本和hdss7-21相同，注意修改一下文件：</p><pre><code class="lang-Bash">[root@hdss7-22 flannel]# vim subnet.env
FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.22.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

flanneld.sh
#!/bin/sh
./flanneld \
  --public-ip=10.4.7.22 \
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 \
  --etcd-keyfile=./cert/client-key.pem \
  --etcd-certfile=./cert/client.pem \
  --etcd-cafile=./cert/ca.pem \
  --iface=ens33 \
  --subnet-file=./subnet.env \
  --healthz-port=2401

chmod +x flanneld.sh

vim /etc/supervisord.d/flannel.ini
[program:flanneld-7-22]
command=/usr/local/flannel/flanneld.sh                       ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/usr/local/flannel                                ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

[root@hdss7-22 flannel]# mkdir -p /data/logs/flanneld/</code></pre><h1>其他方式（没有实验过VxLAN模式）。</h1><p>使用方法：</p><pre><code class="lang-Bash">1、先停止flennel.sh ---通过supervisor stop flanneld-7-[21.22]
2、删除host-gw模型创建的路由
route del -net 172.7.21.0/24 gw 10.4.7.21     hdss7-22上
route del -net 172.7.22.0/24 gw 10.4.7.22     hdss7-21上
3、在etcd节点修改
./etcdctl get /coreos.com/network/config
./etcdctl rm /coreos.com/network/config
etcd]# ./etcdctl set /coreos.com/network/config '{&quot;Network&quot;: &quot;172.7.0.0/16&quot;, &quot;Backend&quot;: {&quot;Type&quot;: &quot;VxLAN&quot;}}'
4、supervisorctl start flanneld-7-21
    supervisorctl start flanneld-7-22
5、查看ifconfig 会多了一个flannel 1的设备，route -n是没有路由的

2.1.17.flannel直接路由模型（智能判定）
类似与mysql日志的mixed模式

./etcdctl set /coreos.com/network/config '{&quot;Network&quot;: &quot;172.7.0.0/16&quot;, &quot;Backend&quot;: {&quot;Type&quot;: &quot;VxLAN&quot;,&quot;Directrouting&quot;: true}}'

[root@hdss7-21 script]# iptables-save |grep -i postrouting
:POSTROUTING ACCEPT [68:4092]
:KUBE-POSTROUTING - [0:0]
-A POSTROUTING -m comment --comment &quot;kubernetes postrouting rules&quot; -j KUBE-POSTROUTING
-A POSTROUTING -s 172.7.21.0/24 ! -o docker0 -j MASQUERADE
-A KUBE-POSTROUTING -m comment --comment &quot;kubernetes service traffic requiring SNAT&quot; -m mark --mark 0x4000/0x4000 -j MASQUERADE

flennel优化
yum install iptables-services  -y
[root@hdss7-21 ~]# systemctl start iptables
[root@hdss7-21 ~]# systemctl enable  iptables

[root@hdss7-21 ~]# iptables -t nat -D POSTROUTING -s 172.7.21.0/24 ! -o docker0 -j MASQUERADE
[root@hdss7-21 ~]# iptables -t nat -I POSTROUTING -s 172.7.21.0/24 ! -d 172.7.0.0/16 ! -o docker0 -j MASQUERADE
[root@hdss7-21 ~]# iptables-save |grep -i postrouting

iptables -t filter -D INPUT -j REJECT --reject-with icmp-host-prohibited
 iptables -t filter -D FORWARD -j REJECT --reject-with icmp-host-prohibited
##########规则定义#########
10.4.7.21主机上的，来源是172.7.21.0/24段的docker的ip，目标ip不是172.7.0.0/16段，网络发包不从docker0桥设备出站的，才进行SNAT转换

~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]

各自访问对方节点，并查看nginx-access日志，可看到现在暴露的都是容器ip
[root@hdss7-21 ~]#  kubectl  logs -f nginx-ds-jtn62
172.7.22.2 - - [23/Nov/2019:17:46:48 +0000] &quot;GET / HTTP/1.1&quot; 200 612 &quot;-&quot; &quot;curl/7.38.0&quot; &quot;-&quot;
[root@hdss7-22 ~]# kubectl  logs -f nginx-ds-d5kl8
10.4.7.21 - - [23/Nov/2019:17:01:37 +0000] &quot;GET / HTTP/1.1&quot; 200 612 &quot;-&quot; &quot;curl/7.38.0&quot; &quot;-&quot;

172.7.21.2 - - [23/Nov/2019:17:43:34 +0000] &quot;GET / HTTP/1.1&quot; 200 612 &quot;-&quot; &quot;curl/7.38.0&quot; &quot;-&quot;

[root@hdss7-21 ~}#route add -net 172.7.22.0/24 gw 10.4.7.22 dev eth0
[root@hdss7-22~}#route add -net 172.7.21.0/24 gw 10.4.7.21 dev eth0
[root@hdss7-21 flannel]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.7.254      0.0.0.0         UG    100    0        0 eth0
10.4.7.0        0.0.0.0         255.255.255.0   U     100    0        0 eth0
172.7.21.0      0.0.0.0         255.255.255.0   U     0      0        0 docker0
172.7.22.0      10.4.7.22       255.255.255.0   UG    0      0        0 eth0

[root@hdss7-22 flannel]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.4.7.254      0.0.0.0         UG    100    0        0 eth0
10.4.7.0        0.0.0.0         255.255.255.0   U     100    0        0 eth0
172.7.21.0      10.4.7.21       255.255.255.0   UG    0      0        0 eth0
172.7.22.0      0.0.0.0         255.255.255.0   U     0      0        0 docker0

注意还要优化一条iptables规则：
~]#  iptables -t filter -I FORWARD -d 172.7.21.0/24 -j ACCEPT

flennel优化
yum install iptables-services  -y
[root@hdss7-22 ~]# systemctl start iptables
[root@hdss7-22 ~]# systemctl enable  iptables

[root@hdss7-22 ~]# iptables -t nat -D POSTROUTING -s 172.7.22.0/24 ! -o docker0 -j MASQUERADE
[root@hdss7-22 ~]# iptables -t nat -I POSTROUTING -s 172.7.22.0/24 ! -d 172.7.0.0/16 ! -o docker0 -j MASQUERADE
[root@hdss7-22 ~]# iptables-save |grep -i postrouting

iptables -t filter -D INPUT -j REJECT --reject-with icmp-host-prohibited
 iptables -t filter -D FORWARD -j REJECT --reject-with icmp-host-prohibited
##########规则定义#########
10.4.7.22主机上的，来源是172.7.22.0/24段的docker的ip，目标ip不是172.7.0.0/16段，网络发包不从docker0桥设备出站的，才进行SNAT转换

~]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[  OK  ]</code></pre><p>各自访问对方节点，并查看nginx-access日志，可看到现在暴露的都是容器ip<br>这里的需要注意防火墙的放通问题。</p>

二进制安装k8s集群-部署flannel(十一)

拷贝证书，需要与etcd通信并配置，所以需要拷贝以下证书

设置flannel的网络插件模式为host-gw

其他方式（没有实验过VxLAN模式）。

Leave a Comment Cancel reply
使用cookie技术保留您的个人信息以便您下次快速评论，继续评论表示您已同意该条款

ARM处理器

linux mint 19 更换国内源的方法！

局域网最简单的DDOS攻击

全球最佳100部经典电影排行推荐

linux mint解锁网易云音乐某某原因不能播放歌曲

百度启用全站HTTPS安全服务

Win10新预览版发布,斯巴达浏览器登场

二进制安装k8s集群-准备部署etcd集群(五)

shell简单的read和if练习

H3C添加端口映射

二进制安装k8s集群-部署flannel(十一)

拷贝证书，需要与etcd通信并配置，所以需要拷贝以下证书

设置flannel的网络插件模式为host-gw

其他方式（没有实验过VxLAN模式）。

Leave a Comment Cancel reply 使用cookie技术保留您的个人信息以便您下次快速评论，继续评论表示您已同意该条款

二进制安装k8s集群-部署flannel(十一)

Leave a Comment Cancel reply
使用cookie技术保留您的个人信息以便您下次快速评论，继续评论表示您已同意该条款