docker的supervisor与inotifywait的使用技巧

在测试环境配置中，会经常使用到docker-compose，当配置文件被挂载到宿主机后，修改配置文件后不能生效就成了问题，鉴于此，我们使用supervisor和inotifywait来使“在宿主机修改配置文件，容器内生效”的作用

supervisor和inotifywait说明

• supervisor

supervisor的配置文件相对简单，他可以在程序期待失败后自动重启，也可做自动的日志轮询，可以单独控制，也可成组的控制。提供本地或远程命令行和Web界面。supervisord通过fork / exec启动其子进程，子进程不进行守护。当进程终止时，操作系统会立即向Supervisor发出信号，这与某些依赖麻烦的PID文件和定期轮询重新启动失败进程的解决方案不同。除了Windows之外，Supervisor几乎可以处理所有事情。它在Linux，Mac OS X，Solaris和FreeBSD上经过测试和支持。它完全用Python编写，因此安装不需要C编译器。参考：http://supervisord.org/我们需要配置supervisor启动docker，在启动supervisor的配置文件中在加入inotifywait来捕获应用程序的文件状态，如果有变化就重新启动一次。介绍下inotifywait

• inotifywait

inotifywait使用Linux的inotify接口有效地等待文件的更改 。它适用于等待shell脚本中的文件更改。它可以在事件发生时退出，也可以在事件发生时继续执行和输出事件。

inotify

inotify- 监视文件系统事件inotify的 API提供了用于监控文件系统事件的机制。Inotify可用于监视单个文件或监视目录。监视目录时，inotify将返回目录本身以及目录内文件的事件。如果你有兴趣你可以参考这里：https://linux.die.net/man/1/inotifywait

示例

我们创建一个容器应用maxscale(这个应用不是很恰当)进行测试，我会公布所有的配置细节pip install supervisor直接安装，然后下载配置文件，在启动脚本中下载配置文件并启动

编写Dockerfile和启动脚本

Dockerfile

FROM centos:7
MAINTAINER wwww.linuxea.com
ENV NODE_VERSION 2.1.9
ENV MAXCON "/data/maxscale/maxscale.cnf"
RUN groupadd -r -g 307 maxscale 
    && useradd -r -g 307 -u 307 maxscale -s /sbin/nologin -M 
    && curl -Lk https://downloads.mariadb.com/MaxScale/${NODE_VERSION}/centos/7/x86_64/maxscale-${NODE_VERSION}.centos.7.tar.gz -o ./maxscale-${NODE_VERSION}.centos.7.tar.gz 
    && tar xf maxscale-${NODE_VERSION}.centos.7.tar.gz -C /usr/local/ 
    && cd /usr/local/ && ln -s maxscale-${NODE_VERSION}.centos.7 maxscale 
    && ln -s /usr/local/maxscale/bin/maxadmin /usr/bin/ 
    && yum -y install epel* && yum install python-pip inotify-tools -y 
    && pip install supervisor 
    && curl -Lk https://raw.githubusercontent.com/LinuxEA-Mark/maxscale/master/conf/supervisord.conf -o /etc/supervisord.conf 
    && yum clean all && rm -rf /maxscale-${NODE_VERSION}.centos.7.tar.gz 
    && curl -Lk https://raw.githubusercontent.com/LinuxEA-Mark/maxscale/master/conf/start.sh -o /start.sh && chmod +x /start.sh
ENTRYPOINT [ "/start.sh" ]

supervisor配置文件

如果有create,delete,modify,move,attrib则会删除pid文件后启动

command=bash -c 'while inotifywait -q -r -e create,delete,modify,move,attrib --exclude "/." /data/maxscale/maxscale.cnf; do rm -rf /data/maxscale/pid/maxscale.pid && /usr/local/maxscale/bin/maxscale -f /data/maxscale/maxscale.cnf '--nodaemon'; done'

完整的配置如下

[unix_http_server]
file=/tmp/supervisor.sock   ; the path to the socket file
[supervisord]
logfile=/tmp/supervisord.log ; main log file; default $CWD/supervisord.log
logfile_maxbytes=50MB        ; max main logfile bytes b4 rotation; default 50MB
logfile_backups=10           ; # of main logfile backups; 0 means none, default 10
loglevel=info                ; log level; default info; others: debug,warn,trace
pidfile=/tmp/supervisord.pid ; supervisord pidfile; default supervisord.pid
nodaemon=false               ; start in foreground if true; default false
minfds=1024                  ; min. avail startup file descriptors; default 1024
minprocs=200                 ; min. avail process descriptors;default 200

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[inet_http_server]
port = 9001
username = user
password = pass 

[program:maxscale]
command=/bin/bash -c "exec /usr/local/maxscale/bin/maxscale -f /data/maxscale/maxscale.cnf '--nodaemon'"
autostart=true
autorestart=false
startretries=0
stdout_events_enabled=true
stderr_events_enabled=true

[program:maxscale_conf]
command=bash -c 'while inotifywait -q -r -e create,delete,modify,move,attrib --exclude "/." /data/maxscale/maxscale.cnf; do rm -rf /data/maxscale/pid/maxscale.pid && /usr/local/maxscale/bin/maxscale -f /data/maxscale/maxscale.cnf '--nodaemon'; done'

start.sh

在脚本中简单的判断了启动的文件 ，如果没有就下载，最后启动

#!/bin/bash
#########################################################################
# File Name: start.sh
# Author: www.linuxea.com
# Email: admin#linuxea.com
# Version: 1.0
# Created Time: 2018年07月14日 星期六 17时45分43秒
#########################################################################
MSE=`ps aux|grep -v grep|grep -c /usr/local/maxscale/bin/maxscale`
MSEID=`ps aux|awk  '/maxscale/{print $2}'|awk NR==1`
MAXCON=/data/maxscale/maxscale.cnf
SVDPATH=/etc/supervisord.conf
if [ -f ${MAXCON} ]; then
    echo 'Maxscale Configuration File Already Exists /data/maxscale/maxscale.cnf '
else 
    echo 'Maxscale Configuration File is Downloading......'
    cd /data/maxscale && mkdir -p {data,cache,logs,tmp,pid} && mkdir -p logs/{binlog,trace}
    chown -R maxscale:maxscale /data/maxscale
    curl -Lk https://raw.githubusercontent.com/LinuxEA-Mark/maxscale/master/maxscale.cnf -o ${MAXCON}
fi    
if [ -f ${SVDPATH} ]; then
  echo 'supervisord Configuration File Already Exists /etc/supervisord.conf '
else
    echo 'Maxscale Process exists, has just been restarted'    
    curl -Lk https://raw.githubusercontent.com/LinuxEA-Mark/maxscale/master/conf/supervisord.conf -o ${SVDPATH}
fi
rm -r /data/maxscale/pid/*
supervisord -n -c /etc/supervisord.conf

dockerfile直接build后，可以推送到dockerhub上

push到docker hub

修改tag

[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ docker tag maxscale:2.1.9 marksugar/maxscale:2.1.9

登陆

[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ docker login 
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username: marksugar
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

开始push

[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ docker push  marksugar/maxscale:2.1.9
The push refers to repository [docker.io/marksugar/maxscale]
d696af8c48b2: Pushed 
abbcf767e0a8: Pushed 
bcc97fbfc9e1: Mounted from library/centos 
2.1.9: digest: sha256:1a449e063e7c26ead421a9bb63eeb49d189a1f6e7303bc1ad9279830e33ad00f size: 948
[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ 

编写docker-compose

[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ cat docker-compose.yaml 
version: '2'
services:
  maxscale:
    image: marksugar/maxscale:2.1.9
    container_name: maxscale
    restart: always
    network_mode: "host"
    volumes: 
    - /data/maxscale:/data/maxscale
    ports:
    - "4006"
[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ 

启动

[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ docker-compose -f /data/maxscale/docker-compose.yaml up -d
Creating maxscale ... done
[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ docker ps -a
CONTAINER ID        IMAGE                      COMMAND             CREATED             STATUS                           PORTS               NAMES
a48f352d7f72        marksugar/maxscale:2.1.9   "/start.sh"         2 seconds ago       Up 2 seconds                                         maxscale
ef49b490ed8d        sonarqube:6.7.4            "./bin/run.sh"      2 days ago          Exited (143) About an hour ago                       sonarqube

查看日志

[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ docker logs  maxscale 
Maxscale Configuration File Already Exists /data/maxscale/maxscale.cnf 
supervisord Configuration File Already Exists /etc/supervisord.conf 
2018-07-14 08:28:01,780 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2018-07-14 08:28:01,790 INFO RPC interface 'supervisor' initialized
2018-07-14 08:28:01,790 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2018-07-14 08:28:01,790 INFO supervisord started with pid 16
2018-07-14 08:28:02,794 INFO spawned: 'maxscale_conf' with pid 19
2018-07-14 08:28:02,796 INFO spawned: 'maxscale' with pid 20
2018-07-14 08:28:03,810 INFO success: maxscale_conf entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2018-07-14 08:28:03,810 INFO success: maxscale entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

查看端口

[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ ss -tlnp
State       Recv-Q Send-Q                                                 Local Address:Port                                                                Peer Address:Port              
LISTEN      0      1024                                                               *:9001                                                                           *:*                   users:(("supervisord",pid=15240,fd=4))
LISTEN      0      32768                                                              *:6603                                                                           *:*                   users:(("maxscale",pid=15245,fd=12))
LISTEN      0      128                                                                *:22992                                                                          *:*                   users:(("sshd",pid=1024,fd=3))
LISTEN      0      100                                                        127.0.0.1:25                                                                             *:*                   users:(("master",pid=1181,fd=13))
LISTEN      0      128                                                        127.0.0.1:6010                                                                           *:*                   users:(("sshd",pid=3547,fd=8))
LISTEN      0      128                                                        127.0.0.1:6011                                                                           *:*                   users:(("sshd",pid=3547,fd=10))
LISTEN      0      32768                                                              *:4006                                                                           *:*                   users:(("maxscale",pid=15245,fd=11))
[root@Linuxea-VM-Node_10_10_240_145 /data/maxscale]$ 

打开web界面看看由于我们在之前的配置文件中已经设置了

[inet_http_server]
port = 9001
username = user
password = pass 

在弹出的登录框输入user pass即可登录