如下图所示
我们先查看下pod的调度情况，分别是10.11.1.45和10.11.0.10在不同的两个节点，并且是不同的网段
- 不同的网段，那么久需要查找路由表，看路由信息
[root@master1 ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
marksugar-deployment-578cdd567b-968gg 1/1 Running 4 8d 10.11.1.45 node1
marksugar-deployment-578cdd567b-fw5rh 1/1 Running 4 8d 10.11.1.48 node1
marksugar-deployment-578cdd567b-nfhtt 1/1 Running 8 12d 10.11.0.10 master1里面分别有两个网卡，分别是eth0和lo
- pod尾968gg开头的mac地址是
9e:66:19:aa:f6:c7，ip是10.11.1.45
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-968gg -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
3: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default
link/ether 9e:66:19:aa:f6:c7 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.11.1.45/24 brd 10.11.1.255 scope global eth0
valid_lft forever preferred_lft forever- pod尾nfhtt开头的mac地址是
ee:a2:33:a2:b6:69，ip是10.11.0.10
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt -- ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
3: eth0@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default
link/ether ee:a2:33:a2:b6:69 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.11.0.10/24 brd 10.11.0.255 scope global eth0
valid_lft forever preferred_lft forever路由匹配原则
我们在上面知道，不同的网段通讯，需要查找路由表，看路由信息，我们就可以查询下路由信息
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-968gg -- route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.11.1.1 0.0.0.0 UG 0 0 0 eth0
10.11.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.244.0.0 10.11.1.1 255.255.0.0 UG 0 0 0 eth0- 路由匹配是最长匹配原则，大致意思是，越精确的信息优先
路由匹配是最长匹配原则，大致意思是，越精细的信息优先，什么意思呢如果是一个ip就优先匹配，而后在到0.0.0.0的默认路由，如果有就走，如过没有就丢掉
当我们ping的时候，就会走默认路由，将信息发送到网关，也就是10.11.1.1
要ping通，就需要源ip，目标ip，源mac，目标mac
路由转发前提
路由转发中ip是不变的，mac地址每经过一条都会发送变化也就是说源ip和目标ip是不发生改变的，但是源mac和目的mac是一直在变的
如上，10.11.1.45的路由表的信息的下一条是10.11.1.1，他们属于同一个网段，因此只需要解析到二层即可。解析的mac地址在宿主机上是ee:e9:19:55:93:d1，如下
7: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default qlen 1000
link/ether ee:e9:19:55:93:d1 brd ff:ff:ff:ff:ff:ff
inet 10.11.1.1/24 brd 10.11.1.255 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::ece9:19ff:fe55:93d1/64 scope link
valid_lft forever preferred_lft forever开始抓包
我们进行抓包来查看两个pod的mac地址是不是和我们上面说的那样。
- 不同节点的pod网络通讯ip不变，而mac地址一直在发生变化
我们在master节点抓nfhtt的包，nfhtt是10.11.0.10的ip，mac地址是ee:a2:33:a2:b6:69
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt -- tcpdump -n -e -i eth0我们并且上面开始Ping，ping的是968gg的pod，ip地址是10.11.1.45，mac地址是9e:66:19:aa:f6:c7
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt -- ping 10.11.1.45
PING 10.11.1.45 (10.11.1.45): 56 data bytes
64 bytes from 10.11.1.45: seq=0 ttl=60 time=0.613 ms
64 bytes from 10.11.1.45: seq=1 ttl=60 time=0.257 ms
64 bytes from 10.11.1.45: seq=2 ttl=60 time=0.206 ms
64 bytes from 10.11.1.45: seq=3 ttl=60 time=0.235 ms
^C
--- 10.11.1.45 ping statistics ---
13 packets transmitted, 13 packets received, 0% packet loss
round-trip min/avg/max = 0.191/0.254/0.613 ms查看抓包的结果
[root@master1 ~]# kubectl exec -it marksugar-deployment-578cdd567b-nfhtt -- tcpdump -n -e -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
06:56:41.779184 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype IPv4 (0x0800), length 98: 10.11.0.10 > 64
06:56:41.779377 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype IPv4 (0x0800), length 98: 10.11.1.45 > 4
06:56:42.779249 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype IPv4 (0x0800), length 98: 10.11.0.10 > 64
06:56:42.779448 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype IPv4 (0x0800), length 98: 10.11.1.45 > 4
06:56:43.779308 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype IPv4 (0x0800), length 98: 10.11.0.10 > 64
06:56:43.779500 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype IPv4 (0x0800), length 98: 10.11.1.45 > 4
06:56:44.042429 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype ARP (0x0806), length 42: Request who-ha
06:56:44.042437 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype ARP (0x0806), length 42: Request who-ha
06:56:44.042440 ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91, ethertype ARP (0x0806), length 42: Reply 10.11.0.
06:56:44.042463 a2:a2:7f:a0:0d:91 > ee:a2:33:a2:b6:69, ethertype ARP (0x0806), length 42: Reply 10.11.0.
^C
10 packets captured
10 packets received by filter
0 packets dropped by kernel查看抓包的结果
我们现在在看下cni0的网关的ip和mac地址如下
- cni0在这里表示的是pod网络的网关
10.11.1.1网段
7: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default qlen 1000
link/ether ee:e9:19:55:93:d1 brd ff:ff:ff:ff:ff:ff
inet 10.11.1.1/24 brd 10.11.1.255 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::ece9:19ff:fe55:93d1/64 scope link
valid_lft forever preferred_lft forever和10.11.0.1网段
7: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1472 qdisc noqueue state UP group default qlen 1000
link/ether a2:a2:7f:a0:0d:91 brd ff:ff:ff:ff:ff:ff
inet 10.11.0.1/24 brd 10.11.0.255 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::a0a2:7fff:fea0:d91/64 scope link
valid_lft forever preferred_lft forever如上，我们看到抓包结果中两个mac地址是ee:a2:33:a2:b6:69 > a2:a2:7f:a0:0d:91
pod尾968gg开头的mac地址是
9e:66:19:aa:f6:c7，ip是10.11.1.45pod尾nfhtt开头的mac地址是ee:a2:33:a2:b6:69，ip是10.11.0.10
我们从 10.11.0.10 ping 10.11.1.45， 10.11.0.10 的mac地址是ee:a2:33:a2:b6:69 ，10.11.1.45的pod的mac地址是9e:66:19:aa:f6:c7，而抓包走的则是10.11.0.10 的mac地址是ee:a2:33:a2:b6:69和10.11.0.1的cni0网卡的mac地址a2:a2:7f:a0:0d:91,返回的也是a2:a2:7f:a0:0d:91和ee:a2:33:a2:b6:69
我们通过pod nfhtt（ee:a2:33:a2:b6:69 10.11.0.10）ping pod为9688GG的(9e:66:19:aa:f6:c7 10.11.1.45)ip而抓包显示，返回的信息的mac地址是10.11.0.1的mac地址。而10.11.0.1是cni0网关
这篇延续上面几篇基础
