k8s基础应用一：docker和kubernetes环境搭建-每日运维

1 macOS安装docker

1.1 安装

macOS一般安装desktop桌面版，用于平时测试。从docker官网下载docker.dmg安装即可。安装后启动docker就可以使用了。

% docker version
Client:
 Cloud integration: v1.0.35+desktop.4
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:28:49 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.23.0 (120376)
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:36 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

1.2 docker dashboard和配置

docker.dmg安装完后，启动docker，打开docker的dashboard可以查看docker的运行情况，也可以在Settings中根据需要修改docker配置：

一般会把memory设置为大于4G，

同时由于国内从国外docker仓库下载镜像一般很慢，一般添加一些国内镜像地址进行加速

文本内容如下：

{
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "20GB"
    }
  },
  "registry-mirrors": [
    "https://registry.cn-hangzhou.aliyuncs.com",
    "https://mirror.ccs.tencentyun.com",
    "https://05f073ad3c0010ea0f4bc00b7105ec20.mirror.swr.myhuaweicloud.com",
    "https://registry.docker-cn.com",
    "http://hub-mirror.c.163.com",
    "http://f1361db2.m.daocloud.io"
  ],
  "debug": true,
  "experimental": false
}

查看docker-for-mac对应的kubernetes版本，并使能kubernetes，这样启动docker时会同时启动kubernetes

1.3 手动下载kubernetes镜像

docker desktop同时可以运行kubernetes，但是由于docker.dmg中并不包含kubernetes相关的组件，需要通过网络进行下载，又因为从国外下载的速度往往非常慢，导致kubernetes往往无法正常启动。这时，我们可以手动下载kubernetes相关镜像。github仓库k8s-for-docker-desktop已经维护了desktop不同kubernetes版本的下载脚本，找到相应版本的images.properties和load_images.sh文件下载到本地，然后执行脚本即可。

sh load_images.sh

load_images.sh脚本的内容如下：

#!/bin/bash

file="images.properties"

if [ -f "$file" ]
then
  echo "$file found."

  while IFS='=' read -r key value
  do
    #echo "${key}=${value}"
    docker pull ${value}
    docker tag ${value} ${key}
    docker rmi ${value}
  done < "$file"

else
  echo "$file not found."
fi

如果你的docker desktop对应的kubernetes版本比较新，k8s-for-docker-desktop仓库还没有维护，可以自己编辑images.properties文件，在文件中添加相应版本的kubernetes镜像，如上面docker对应的v1.27.2版本kubernetes可以使用如下内容的images.properties:

registry.k8s.io/pause:3.8=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.8
registry.k8s.io/kube-controller-manager:v1.27.2=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-controller-manager:v1.27.2
registry.k8s.io/kube-scheduler:v1.27.2=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-scheduler:v1.27.2
registry.k8s.io/kube-proxy:v1.27.2=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-proxy:v1.27.2
registry.k8s.io/kube-apiserver:v1.27.2=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-apiserver:v1.27.2
registry.k8s.io/etcd:3.5.5-0=registry.cn-hangzhou.aliyuncs.com/google_containers/etcd:3.5.5-0
registry.k8s.io/coredns/coredns:v1.9.3=registry.cn-hangzhou.aliyuncs.com/google_containers/coredns:v1.9.3
registry.k8s.io/ingress-nginx/controller:v1.6.4=registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.6.4
registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.5.2=registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.5.2

1.4 docker和k8s测试

上面kubernetes的相关镜像成功下载后，k8s一般就正常启动了，我们查看一下已安装的本地镜像

% docker images
REPOSITORY                                                TAG                                                                          IMAGE ID       CREATED         SIZE
hubproxy.docker.internal:5555/docker/desktop-kubernetes   kubernetes-v1.27.2-cni-v1.2.0-critools-v1.27.0-cri-dockerd-v0.3.2-1-debian   8ba658ef36dd   3 months ago    398MB
registry.k8s.io/kube-apiserver                            v1.27.2                                                                      72c9df6be7f1   4 months ago    115MB
registry.k8s.io/kube-controller-manager                   v1.27.2                                                                      2ee705380c3c   4 months ago    107MB
registry.k8s.io/kube-scheduler                            v1.27.2                                                                      305d7ed1dae2   4 months ago    56.2MB
registry.k8s.io/kube-proxy                                v1.27.2                                                                      29921a084542   4 months ago    66.5MB
docker/desktop-vpnkit-controller                          dc331cb22850be0cdd97c84a9cfecaf44a1afb6e                                     3750dfec169f   4 months ago    35MB
registry.k8s.io/ingress-nginx/controller                  v1.6.4                                                                       bad06a492068   7 months ago    282MB
registry.k8s.io/coredns/coredns                           v1.10.1                                                                      97e04611ad43   7 months ago    51.4MB
sysrepo/sysrepo-netopeer2                                 latest                                                                       976d1dcf6c0c   7 months ago    622MB
registry.k8s.io/etcd                                      3.5.7-0                                                                      24bc64e91103   8 months ago    181MB
registry.k8s.io/pause                                     3.9                                                                          829e9de338bd   11 months ago   514kB
registry.k8s.io/etcd                                      3.5.5-0                                                                      b9a1dfeddea9   12 months ago   179MB
registry.k8s.io/pause                                     3.8                                                                          4e42fb3c9d90   15 months ago   514kB
registry.k8s.io/coredns/coredns                           v1.9.3                                                                       b19406328e70   16 months ago   47.7MB
docker/desktop-storage-provisioner                        v2.0                                                                         c027a58fa0bb   2 years ago     39.8MB
registry.k8s.io/ingress-nginx/kube-webhook-certgen        v1.5.2                                                                       9dadbac53d3b

可以发现就是我们上面下载安装的k8s镜像。查看一下k8s集群信息：

% kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:6443
CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

再看一下k8s系统已经启动的工作节点，只有一个，名字就叫做docker-desktop：

% kubectl get nodes
NAME             STATUS   ROLES           AGE   VERSION
docker-desktop   Ready    control-plane   11d   v1.27.2

查看一下k8s已创建的namespace：

% kubectl get namespaces
NAME              STATUS   AGE
default           Active   11d
kube-node-lease   Active   11d
kube-public       Active   11d
kube-system       Active   11d

查看k8s系统启动的pod，pod都放在kube-system这个namespace下，由此也发现虽然kubernetes本事是一个容器（pod）编排和管理系统，docker-for-mac的kubernetes系统的所有组件本身都是以pod的形式运行的

% kubectl get pod --all-namespaces
NAMESPACE     NAME                                     READY   STATUS    RESTARTS      AGE
kube-system   coredns-5d78c9869d-9f9wp                 1/1     Running   1 (11d ago)   11d
kube-system   coredns-5d78c9869d-kqxgl                 1/1     Running   1 (11d ago)   11d
kube-system   etcd-docker-desktop                      1/1     Running   1 (11d ago)   11d
kube-system   kube-apiserver-docker-desktop            1/1     Running   1 (11d ago)   11d
kube-system   kube-controller-manager-docker-desktop   1/1     Running   1 (11d ago)   11d
kube-system   kube-proxy-brjpm                         1/1     Running   1 (11d ago)   11d
kube-system   kube-scheduler-docker-desktop            1/1     Running   1 (11d ago)   11d
kube-system   storage-provisioner                      1/1     Running   2 (11d ago)   11d
kube-system   vpnkit-controller                        1/1     Running   1 (11d ago)   11d

2 linux安装docker

2.1 通过yum安装

2.1.1 下载docker

如果网络环境允许yum安装，可以直接通过yum -y install docker命令下载

$ sudo su -
# yum -y install docker

yum安装完后，可以在/usr/lib/systemd/system目录下看到以下跟docker相关的服务管理配置文件：

# cd /usr/lib/systemd/system
# ls | grep docker
docker-cleanup.service
docker-cleanup.timer
docker.service
docker-storage-setup.service

查看这些文件的内容，我们可以就可以知道docker涉及的二进制文件和配置文件位置了。

2.1.2 启动dockerd服务

启动docker：

# systemctl daemon-reload
# systemctl start docker

设置开机自启动：

# systemctl daemon-reload
# systemctl enable --now docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.

2.2 二进制包安装

2.2.1 安装

直接通过yum安装的docker，有可能安装的docker版本比较老旧，比如上面安装的docker版本为：

$ docker -v
Docker version 1.13.1, build 7d71120/1.13.1

我们也可以到docker官网自己下载二进制包，手动安装。先查看机器的CPU架构和操作系统版本：

# uname -a
Linux 4.14.0_1-0-0-45 #2 SMP Tue Oct 19 18:27:28 CST 2021 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/centos-release
CentOS Linux release 7.5.1804 (Core)

然后从官网下载对应CPU架构和操作系统的二进制包docker-24.0.6.tgz，然后安装

$ sudo su -
# tar -xzf docker-24.0.6.tgz
# 安装包中的二进制文件
# ls docker
containerd  containerd-shim-runc-v2  ctr  docker  dockerd  docker-init  docker-proxy  runc
# mv docker/* /usr/bin/

然后编辑相应的service配置文件

# cd /usr/lib/systemd/system

# cat  docker.service 
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target
Requires=docker-cleanup.timer

[Service]
Type=notify
NotifyAccess=main
EnvironmentFile=-/run/containers/registries.conf
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash
Environment=DOCKER_HTTP_HOST_COMPAT=1
Environment=PATH=/usr/bin:/usr/sbin
ExecStart=/usr/bin/dockerd 
          $OPTIONS 
          $DOCKER_STORAGE_OPTIONS 
          $DOCKER_NETWORK_OPTIONS 
          $ADD_REGISTRY 
          $BLOCK_REGISTRY 
          $INSECURE_REGISTRY 
          $REGISTRIES
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
Restart=on-abnormal
KillMode=process

[Install]
WantedBy=multi-user.target
EOF

# cat  docker-cleanup.timer 
[Unit]
Description=Run docker-cleanup every hour

[Timer]
OnCalendar=hourly
Persistent=true

[Install]
WantedBy=timers.target
EOF

# cat  docker-cleanup.service
[Unit]
Description=Docker Cleanup
Requires=docker.service


[Service]
Type=oneshot
ExecStart=/usr/bin/sh -c "DEAD=`docker ps -aq -f status=dead` && [ -n "$DEAD" ] && docker rm $DEAD; exit 0"

[Install]
WantedBy=multi-user.target
EOF

docker的镜像等数据默认存放在/var/lib/docker目录中，我们一般希望更改它的存储位置。我们也可能希望设置自己的docker仓库镜像地址，这些配置以及其他的一些相关配置可以在docker的配置文件/etc/docker/daemon.json中指定

# cd /etc/docker
# cat  daemon.json
{
  "insecure-registries": ["registry.cn-hangzhou.aliyuncs.com"],
  "registry-mirrors": ["https://registry.cn-hangzhou.aliyuncs.com"],
  "data-root":"/home/disk1/docker"
}
EOF

配置完成就可以启动了

# systemctl daemon-reload
# systemctl start docker
# systemctl enable --now docker

查看一下安装信息：

# docker version
Client:
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:30:51 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:32:17 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.3
  GitCommit:        7880925980b188f4c97b462f709d0db8e8962aff
 runc:
  Version:          1.1.9
  GitCommit:        v1.1.9-0-gccaecfc
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
# docker info
Client:
 Version:    24.0.6
 Context:    default
 Debug Mode: false

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 2
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7880925980b188f4c97b462f709d0db8e8962aff
 runc version: v1.1.9-0-gccaecfc
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 4.14.0_1-0-0-45
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 48
 Total Memory: 125.3GiB
 Name: szm3b-szlab-dev20.szm3b.baidu.com
 ID: 6bca25b8-7482-421e-993f-93cf787c4d08
 Docker Root Dir: /home/disk1/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  registry.cn-hangzhou.aliyuncs.com
  127.0.0.0/8
 Registry Mirrors:
  https://registry.cn-hangzhou.aliyuncs.com/
 Live Restore Enabled: false
 Product License: Community Engine

2.3 问题

2.3.1 手动创建docker0网桥

如果启动时出现错误：dockerd-current: Error starting daemon: Error initializing network controller: list bridge addresses failed: no available network，则表示没有配置docker0网桥，执行以下命令添加docker0网桥，并配置网桥的ip地址网段：

# ip link add name docker0 type bridge
# ip addr add dev docker0 172.17.0.1/16

执行完后确认docker0网桥已安装

# ip addr
40: docker0:  mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:a5:ce:b1:a5 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
       
# ip link
40: docker0:  mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default 
    link/ether 02:42:a5:ce:b1:a5 brd ff:ff:ff:ff:ff:ff

2.3.2 手动挂载cgroups

如果启动时出现failed to start daemon: Devices cgroup isn‘t mounted错误，可以执行以下脚本进行挂载：

# cat  mount_cgroup.sh
#! /bin/sh
set -e # 这句是告诉bash如何有任何语句执行结果不为ture，就应该退出。

if grep -v '^#' /etc/fstab | grep -q cgroup; then
	echo 'cgroups mounted from fstab, not mounting /sys/fs/cgroup'
	exit 0
fi

# kernel provides cgroups?
if [ ! -e /proc/cgroups ]; then
	exit 0
fi

# 确保目录存在
if [ ! -d /sys/fs/cgroup ]; then
	exit 0
fi

# mount /sys/fs/cgroup if not already done
if ! mountpoint -q /sys/fs/cgroup; then
	mount -t tmpfs -o uid=0,gid=0,mode=0755 cgroup /sys/fs/cgroup
fi

cd /sys/fs/cgroup

# get/mount list of enabled cgroup controllers
for sys in $(awk '!/^#/ { if ($4 == 1) print $1 }' /proc/cgroups); do
	mkdir -p $sys
	if ! mountpoint -q $sys; then
		if ! mount -n -t cgroup -o $sys cgroup $sys; then rmdir $sys || true
		fi
	fi
done
exit 0
EOF

执行完后查看cgroup已经安装：

# df -h
文件系统	      容量  已用  可用 已用%% 挂载点
/dev/sda2              19G  6.8G   12G  38% /
tmpfs                  48G   11M   48G   1% /dev/shm
/dev/sda3              14G  5.1G  8.6G  38% /var
/dev/sda4              14G  1.1G   13G   8% /noah
/dev/sda5             4.6G   68M  4.5G   2% /matrix
/dev/sda6             1.9G   20K  1.9G   1% /has
/dev/sda7             6.4G  9.9M  6.4G   1% /tmp
/dev/sda8             392G  101G  272G  27% /home
/dev/sdb1             1.7T   68M  1.7T   1% /home/disk1
cgroup                 48G     0   48G   0% /sys/fs/cgroup

k8s基础应用一：docker和kubernetes环境搭建

1.1 安装

1.2 docker dashboard和配置

1.3 手动下载kubernetes镜像

1.4 docker和k8s测试

2.1 通过yum安装

2.1.1 下载docker

2.1.2 启动dockerd服务

2.2 二进制包安装

2.2.1 安装

2.3 问题

2.3.1 手动创建docker0网桥

2.3.2 手动挂载cgroups

相关文章

发布评论取消回复

醒在深海的猫

Awesome Project: 探索 MatrixOrigin 云原生分布式数据库

百度搜索：蓝易云 – 熟悉ubuntu apt-get命令详解

oracle 自动撤销

k8s基础应用一：docker和kubernetes环境搭建

1.1 安装

1.2 docker dashboard和配置

1.3 手动下载kubernetes镜像

1.4 docker和k8s测试

2.1 通过yum安装

2.1.1 下载docker

2.1.2 启动dockerd服务

2.2 二进制包安装

2.2.1 安装

2.3 问题

2.3.1 手动创建docker0网桥

2.3.2 手动挂载cgroups

相关文章

发布评论 取消回复

醒在深海的猫

Awesome Project: 探索 MatrixOrigin 云原生分布式数据库

百度搜索：蓝易云 – 熟悉ubuntu apt-get命令详解

oracle 自动撤销

发布评论取消回复