swarm可以轻松的发布服务和端口,所有节点都参与入口的路由网络,路由网络能够使得集群中每个节点都能够接受已经发布端口上的任何服务。即使在节点上没有运行任何服务,也可以在集群中运行任何服务。路由网络将所有传入请求路由到可用节点上,也就是存活的容器上。
扩展阅读:https://docs.docker.com/engine/swarm/how-swarm-mode-works/services/#tasks-and-scheduling
`https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/`
假设我们范围8080端口,群集负载平衡器将求路由到活动容器。
路由网关在发布后的端口上侦听分配给该节点的任何IP地址。对于外部可路由的IP地址,端口可从主机外部使用。对于所有其他IP地址,访问仅在主机内可用。这些在官网有 ,如下图:当访问到192.168.99.102:8080时候不会访问到本机,而会路由器到其他活着的节点上继续提供服务我们配置一个外部的负载均衡器来调度请求到集群,如下图:当应用请求到haproxy会将请求发送到swarm,通过swarm路由到后端web先添加防火墙端口
- 4789UDP用于容器入口网络
-
7946TCP/UDP用于容器网络发现如果端口未开放,达不到冗余的状态
iptables -I INPUT 4 -p udp -m udp -m state --state NEW -m multiport --dports 7946 -m comment --comment "tcp_swarm" -j ACCEPT iptables -I INPUT 4 -p tcp -m tcp -m state --state NEW -m multiport --dports 4789 -m comment --comment "udp_swarm" -j ACCEPT
接着之前的机器进行配置
集群节点
[root@DS-VM-Node117-LinuxEA ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS 3czo94batsbkgmeana39tys6v DS-VM-Node113-LinuxEA.cluster.com Ready Active as4u4yh1h5h84y06h2etad4yb * DS-VM-Node117-LinuxEA.cluster.com Ready Active Leader d464utrj8hgseauht11zddy2i DS-VM-Node98-LinuxEA.cluster.com Ready Active
创建集群
[root@DS-VM-Node117-LinuxEA ~]# docker service create --replicas 4 --name www --publish 8080:81 marksugar/lnp_nginx:1 1y94ii97w9n1yz910my9mik9b
查看
[root@DS-VM-Node117-LinuxEA ~]# docker service ps www ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR dlcax69emtkcs4ja5g45okknb www.1 marksugar/lnp_nginx:1 DS-VM-Node113-LinuxEA.cluster.com Running Running 18 seconds ago 7epjln1ozwzk2mx2vipckw7ci www.2 marksugar/lnp_nginx:1 DS-VM-Node117-LinuxEA.cluster.com Running Running 8 seconds ago e8vzl73at349rvtdx66nvb5jr www.3 marksugar/lnp_nginx:1 DS-VM-Node98-LinuxEA.cluster.com Running Running less than a second ago 0zr4thqnn5bfwy1ion284yjau www.4 marksugar/lnp_nginx:1 DS-VM-Node98-LinuxEA.cluster.com Running Running less than a second ago
修改本地节点的文件做测试
[root@DS-VM-Node117-LinuxEA ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ada3d9638ef0 marksugar/lnp_nginx:1 "/start.sh" 36 seconds ago Up 36 seconds www.2.7epjln1ozwzk2mx2vipckw7ci
我们进入容器添加一个index.html做测试
[root@DS-VM-Node117-LinuxEA ~]# docker exec -it www.2.7epjln1ozwzk2mx2vipckw7ci sh / # echo '10.10.240.117' > /data/wwwroot/index.html / # exit
修改98节点的文件做测试
[root@DS-VM-Node98-LinuxEA ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES be78c3636cc7 marksugar/lnp_nginx:1 "/start.sh" 2 minutes ago Up 2 minutes www.3.e8vzl73at349rvtdx66nvb5jr 4c20f1f0e435 marksugar/lnp_nginx:1 "/start.sh" 2 minutes ago Up 2 minutes www.4.0zr4thqnn5bfwy1ion284yjau [root@DS-VM-Node98-LinuxEA ~]# docker exec -it www.3.e8vzl73at349rvtdx66nvb5jr sh / # echo '10.10.0.98:1' > /data/wwwroot/index.html / # exit [root@DS-VM-Node98-LinuxEA ~]# docker exec -it www.4.0zr4thqnn5bfwy1ion284yjau sh / # echo '10.10.0.98:2' > /data/wwwroot/index.html / # exit
修改113节点的文件做测试
[root@DS-VM-Node113-LinuxEA ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d6320beb5767 marksugar/lnp_nginx:1 "/start.sh" 5 minutes ago Up 5 minutes www.1.dlcax69emtkcs4ja5g45okknb [root@DS-VM-Node113-LinuxEA ~]# docker exec -it www.1.dlcax69emtkcs4ja5g45okknb sh / # echo '10.10.240.113' > /data/wwwroot/index.html / # exit
准备haproxy节点做调度
frontend frontend-web.com bind *:80 mode http option httplog log global default_backend backend-webgroup.com backend backend-webgroup.com option forwardfor header X-REALL-IP option httpchk HEAD / HTTP/1.0 balance roundrobin server web-node1 10.10.0.98:8080 check inter 2000 rise 30 fall 15 server web-node2 10.10.240.117:8080 check inter 2000 rise 30 fall 15 server web-node3 10.10.240.113:8080 check inter 2000 rise 30 fall 15
haproxy测试调度
[root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.240.113 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.240.113 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.0.98:2 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.0.98:1 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.0.98:1 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.240.113 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.240.117 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.240.113 [root@DS-VM-Node49 /data/docker/haproxy]# curl 10.0.1.49 10.10.0.98:2
我们在前面有说过,路由会路由到任何一台发布端口且容器存活的主机(在集群中的机器),我们在任何一台上访问都会路由到其他正常的节点
[root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.0.98:1 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.240.117 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.240.113 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.0.98:1 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.240.117 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.240.113 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.0.98:2 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.0.98:1 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.240.117 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.0.98:2 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.0.98:1 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.240.117 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.240.113 [root@DS-VM-Node113-LinuxEA ~]# curl 10.10.240.113:8080 10.10.0.98:2
我们关掉一个容器测试,仍然可以继续访问
[root@DS-VM-Node113-LinuxEA ~]# docker stop www.1.dlcax69emtkcs4ja5g45okknb www.1.dlcax69emtkcs4ja5g45okknb
这时swarm会检测并且重启up一个新的容器提供服务
[root@DS-VM-Node113 ~]# docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 25221de19c49 marksugar/lnp_nginx:1 "/start.sh" 2 minutes ago Up 2 minutes www.1.enmcaoybhnyq278uccovhlk0h d6320beb5767 marksugar/lnp_nginx:1 "/start.sh" About an hour ago Exited (137) 51 minutes ago www.1.dlcax69emtkcs4ja5g45okknb