Docker swarm集群节点路由网络(3)

应用运维 2023-07-16 向阳逐梦 手机阅读
Scan me!

swarm可以轻松的发布服务和端口，所有节点都参与入口的路由网络，路由网络能够使得集群中每个节点都能够接受已经发布端口上的任何服务。即使在节点上没有运行任何服务，也可以在集群中运行任何服务。路由网络将所有传入请求路由到可用节点上，也就是存活的容器上。

扩展阅读：https://docs.docker.com/engine/swarm/how-swarm-mode-works/services/#tasks-and-scheduling

     `https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/`

假设我们范围8080端口，群集负载平衡器将求路由到活动容器。

路由网关在发布后的端口上侦听分配给该节点的任何IP地址。对于外部可路由的IP地址，端口可从主机外部使用。对于所有其他IP地址，访问仅在主机内可用。这些在官网有 ，如下图：Screenshot1.png当访问到192.168.99.102：8080时候不会访问到本机，而会路由器到其他活着的节点上继续提供服务我们配置一个外部的负载均衡器来调度请求到集群，如下图：Screenshot2.png当应用请求到haproxy会将请求发送到swarm，通过swarm路由到后端web先添加防火墙端口

  • 4789UDP用于容器入口网络

  • 7946TCP/UDP用于容器网络发现如果端口未开放，达不到冗余的状态

    		iptables -I INPUT 4 -p udp -m udp -m state --state NEW -m multiport --dports 7946 -m comment --comment "tcp_swarm" -j ACCEPT
    		iptables -I INPUT 4 -p tcp -m tcp -m state --state NEW -m multiport --dports 4789 -m comment --comment "udp_swarm" -j ACCEPT

    接着之前的机器进行配置

    集群节点

    		[root@DS-VM-Node117-LinuxEA ~]# docker node ls
    		ID                           HOSTNAME                   STATUS  AVAILABILITY  MANAGER STATUS
    		3czo94batsbkgmeana39tys6v    DS-VM-Node113-LinuxEA.cluster.com  Ready   Active        
    		as4u4yh1h5h84y06h2etad4yb *  DS-VM-Node117-LinuxEA.cluster.com  Ready   Active        Leader
    		d464utrj8hgseauht11zddy2i    DS-VM-Node98-LinuxEA.cluster.com   Ready   Active

    创建集群

    		[root@DS-VM-Node117-LinuxEA ~]# docker service create --replicas 4 --name www --publish 8080:81 marksugar/lnp_nginx:1
    		1y94ii97w9n1yz910my9mik9b

    查看

    		[root@DS-VM-Node117-LinuxEA ~]# docker service ps www
    		ID                         NAME   IMAGE                  NODE                       DESIRED STATE  CURRENT STATE                   ERROR
    		dlcax69emtkcs4ja5g45okknb  www.1  marksugar/lnp_nginx:1  DS-VM-Node113-LinuxEA.cluster.com  Running        Running 18 seconds ago          
    		7epjln1ozwzk2mx2vipckw7ci  www.2  marksugar/lnp_nginx:1  DS-VM-Node117-LinuxEA.cluster.com  Running        Running 8 seconds ago           
    		e8vzl73at349rvtdx66nvb5jr  www.3  marksugar/lnp_nginx:1  DS-VM-Node98-LinuxEA.cluster.com   Running        Running less than a second ago  
    		0zr4thqnn5bfwy1ion284yjau  www.4  marksugar/lnp_nginx:1  DS-VM-Node98-LinuxEA.cluster.com   Running        Running less than a second ago

    修改本地节点的文件做测试

    		[root@DS-VM-Node117-LinuxEA ~]# docker ps -a
    		CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS              PORTS               NAMES
    		ada3d9638ef0        marksugar/lnp_nginx:1   "/start.sh"         36 seconds ago      Up 36 seconds                           www.2.7epjln1ozwzk2mx2vipckw7ci

    我们进入容器添加一个index.html做测试

    		[root@DS-VM-Node117-LinuxEA ~]# docker exec -it www.2.7epjln1ozwzk2mx2vipckw7ci sh
    		/ # echo '10.10.240.117' > /data/wwwroot/index.html
    		/ # exit

    修改98节点的文件做测试

    		[root@DS-VM-Node98-LinuxEA ~]# docker ps -a
    		CONTAINER ID        IMAGE                   COMMAND             CREATED             STATUS                     PORTS               NAMES
    		be78c3636cc7        marksugar/lnp_nginx:1   "/start.sh"         2 minutes ago       Up 2 minutes                                   www.3.e8vzl73at349rvtdx66nvb5jr
    		4c20f1f0e435        marksugar/lnp_nginx:1   "/start.sh"         2 minutes ago       Up 2 minutes                                   www.4.0zr4thqnn5bfwy1ion284yjau
    		[root@DS-VM-Node98-LinuxEA ~]# docker exec -it www.3.e8vzl73at349rvtdx66nvb5jr sh
    		/ # echo '10.10.0.98:1' > /data/wwwroot/index.html
    		/ # exit
    		[root@DS-VM-Node98-LinuxEA ~]# docker exec -it www.4.0zr4thqnn5bfwy1ion284yjau sh
    		/ # echo '10.10.0.98:2' > /data/wwwroot/index.html
    		/ # exit

    修改113节点的文件做测试

    		[root@DS-VM-Node113-LinuxEA ~]# docker ps -a
    		CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS               NAMES
    		d6320beb5767        marksugar/lnp_nginx:1   "/start.sh"              5 minutes ago       Up 5 minutes                            www.1.dlcax69emtkcs4ja5g45okknb
    		[root@DS-VM-Node113-LinuxEA ~]# docker exec -it www.1.dlcax69emtkcs4ja5g45okknb sh
    		/ # echo '10.10.240.113' > /data/wwwroot/index.html
    		/ # exit

    准备haproxy节点做调度

    		frontend frontend-web.com
    		      bind *:80
    		      mode http
    		      option httplog
    		      log global
    		      default_backend backend-webgroup.com
    		      
    		backend backend-webgroup.com
    		      option forwardfor header X-REALL-IP
    		      option httpchk HEAD / HTTP/1.0
    		      balance roundrobin 
    		      server web-node1 10.10.0.98:8080 check inter 2000 rise 30 fall 15
    		      server web-node2 10.10.240.117:8080 check inter 2000 rise 30 fall 15
    		      server web-node3 10.10.240.113:8080 check inter 2000 rise 30 fall 15

    haproxy测试调度

    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.240.113
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.240.113
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.0.98:2
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.0.98:1
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.0.98:1
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.240.113
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.240.117
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.240.113
    		[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49
    		10.10.0.98:2

    我们在前面有说过，路由会路由到任何一台发布端口且容器存活的主机(在集群中的机器)，我们在任何一台上访问都会路由到其他正常的节点

    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.0.98:1
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.240.117
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.240.113
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.0.98:1
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.240.117
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.240.113
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.0.98:2
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.0.98:1
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.240.117
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.0.98:2
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.0.98:1
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.240.117
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.240.113
    		[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080
    		10.10.0.98:2

    我们关掉一个容器测试，仍然可以继续访问

    		[root@DS-VM-Node113-LinuxEA ~]# docker stop www.1.dlcax69emtkcs4ja5g45okknb 
    		www.1.dlcax69emtkcs4ja5g45okknb

    这时swarm会检测并且重启up一个新的容器提供服务

    		[root@DS-VM-Node113 ~]# docker ps -a
    		CONTAINER ID        IMAGE                   COMMAND           CREATED             STATUS                        PORTS   NAMES
    		25221de19c49        marksugar/lnp_nginx:1   "/start.sh"       2 minutes ago       Up 2 minutes                          www.1.enmcaoybhnyq278uccovhlk0h
    		d6320beb5767        marksugar/lnp_nginx:1   "/start.sh"       About an hour ago   Exited (137) 51 minutes ago           www.1.dlcax69emtkcs4ja5g45okknb