Kubernetes 二进制升级

云计算 2023-05-04 大猫手机阅读

Kubernetes集群升级非常简单，小版本基本上是只需要更新二进制文件即可。如果大版本升级需要注意kubelet参数的变化，以及其他组件升级之后的变化。由于Kubernetes版本更新过快许多依赖并没有解决完善，并不建议生产环境使用较新版本
这里二进制安装参考下面文章，文件路径都给予下面的文章，如果有不相同的地方可以自行修改即可。有问题请在下方提问！

    <img src="https://img.mryunwei.com/uploads/2023/05/20230504005646993.png" alt>

        <img src="https://img.mryunwei.com/uploads/2023/05/20230504005646993.png" alt>

Kubernetes 1.14 二进制集群安装

                <img src="https://img.mryunwei.com/uploads/2023/05/20230504085651931.gif">新闻联播老司机

19年8月13日

喜欢：1

浏览：18.6k

本次升级的环境版本为1.14升级到1.15版本在每个大版本中，k8s组件的配置文件以及相关参数可能会发生变化，修改好对应的参数问题一般影响不大。官方文档下载地址:https://github.com/kubernetes/kubernetes/releases API版本变更适配对于不同版本的Kubernetes，许多资源对象的API的版本可能会变更。可以通过下面的命令获取当前的API信息

[root@k8s-01 ~]# kubectl explain deployment.apiVersion
KIND:     Deployment
VERSION:  extensions/v1beta1

FIELD:    apiVersion 

DESCRIPTION:
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#resources

升级kubectl

备份kubectl，在所有的master节点操作

cd /opt/k8s/bin
[root@k8s-01 bin]# mv kubectl{,.bak_2020-04-13}

接下来我们下载最新的二进制包

mkdir -p /opt/new-client
wget -P /opt/new-client http://down.i4t.com/k8s1.15/kubernetes-client-linux-amd64.tar.gz
cd /opt/new-client
tar xf kubernetes-client-linux-amd64.tar.gz

分发kubectl到所有节点

cd /opt/new-client
source /opt/k8s/bin/environment.sh
for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kubernetes/client/bin/kubectl root@${node_ip}:/opt/k8s/bin/
    ssh root@${node_ip} "chmod +x /opt/k8s/bin/*"
  done

升级前

[root@k8s-01 bin]# kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T16:23:09Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T16:14:56Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

升级后因为apiserver还没有升级，所以在Server Version中显示为1.14版本

[root@k8s-01 new]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.11", GitCommit:"d94a81c724ea8e1ccc9002d89b7fe81d58f89ede", GitTreeState:"clean", BuildDate:"2020-03-12T21:08:59Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.2", GitCommit:"66049e3b21efe110454d67df4fa62b08ea79a19b", GitTreeState:"clean", BuildDate:"2019-05-16T16:14:56Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

升级master组件

kubectl命令行工具已经升级完毕，接下来我们升级master (这里使用测试环境进行演示，线上环境可以一台一台的升级替换)

#在所有master节点执行 (升级线上环境可以不停止keeplived)
systemctl stop keepalived   # 先停掉本机keepalived，切走高可用VIP地址
systemctl stop kube-apiserver
systemctl stop kube-scheduler
systemctl stop kube-controller-manager

备份旧版本

for i in k8s-01 k8s-02 k8s-03  #这里只需要master节点，因为现在升级是master节点
do
  ssh root@$i mv /opt/k8s/bin/kube-apiserver{,.bak_2020-04-13}
  ssh root@$i mv /opt/k8s/bin/kube-controller-manager{,.bak_2020-04-13}
  ssh root@$i mv /opt/k8s/bin/kube-scheduler{,.bak_2020-04-13}
  ssh root@$i mv /opt/k8s/bin/kubeadm{,.bak_2020-04-13}
done

升级拷贝新版本

mkdir /opt/new-server-k8s
wget -P /opt/new-server-k8s/ down.i4t.com/k8s1.15/kubernetes-server-linux-amd64.tar.gz
cd /opt/new-server-k8s
tar xf kubernetes-server-linux-amd64.tar.gz

#复制新版本客户端
cd /opt/new-server-k8s
source /opt/k8s/bin/environment.sh
for node_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kubernetes/server/bin/kube-apiserver root@${node_ip}:/opt/k8s/bin/
    scp kubernetes/server/bin/{kube-controller-manager,kube-scheduler,kubeadm} root@${node_ip}:/opt/k8s/bin/
    ssh root@${node_ip} "chmod +x /opt/k8s/bin/*"
  done

接下来在所有master启动keepalived和apiserver

#所有master节点启动
systemctl start keepalived
systemctl start kube-apiserver

启动时可以通过下面的命令查看服务是否有不兼容的问题

journalctl -fu kube-apiserver

可以查看到etcd中的数据说明kube-apiserver没有问题

#这里目前只有apiserver数据恢复，所以只要能出结果说明apiserver正常
[root@k8s-01 new-server-k8s]# kubectl  get cs
NAME                 STATUS      MESSAGE                                                                                     ERROR
scheduler            Unhealthy   Get http://127.0.0.1:10251/healthz: dial tcp 127.0.0.1:10251: connect: connection refused   
controller-manager   Unhealthy   Get http://127.0.0.1:10252/healthz: dial tcp 127.0.0.1:10252: connect: connection refused   
etcd-1               Healthy     {"health":"true"}                                                                           
etcd-2               Healthy     {"health":"true"}                                                                           
etcd-0               Healthy     {"health":"true"}

并且查看客户端和服务端的版本都是v1.15.11，说明版本相同没有问题

[root@k8s-01 new-server-k8s]# kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.11", GitCommit:"d94a81c724ea8e1ccc9002d89b7fe81d58f89ede", GitTreeState:"clean", BuildDate:"2020-03-12T21:08:59Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.11", GitCommit:"d94a81c724ea8e1ccc9002d89b7fe81d58f89ede", GitTreeState:"clean", BuildDate:"2020-03-12T21:00:06Z", GoVersion:"go1.12.17", Compiler:"gc", Platform:"linux/amd64"}

接下来启动其他组件

systemctl start kube-controller-manager && systemctl start kube-scheduler

#所有master节点执行

查看启动状态,此时kubernetes集群已经恢复

[root@k8s-01 ~]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-1               Healthy   {"health":"true"}   
etcd-2               Healthy   {"health":"true"}   
etcd-0               Healthy   {"health":"true"}

升级node组件

现在需要停止服务，并且备份二进制文件

#在所有node节点执行
systemctl stop kubelet
systemctl stop kube-proxy

所有node节点备份二进制文件

for i in k8s-01 k8s-02 k8s-03 k8s-04 k8s-05
do
  ssh root@$i mv /opt/k8s/bin/kubelet{,.bak_2020-04-13}
  ssh root@$i mv /opt/k8s/bin/kube-proxy{,.bak_2020-04-13}
done

现在拷贝新的二进制文件

cd /opt/new-server-k8s
source /opt/k8s/bin/environment.sh
for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    scp kubernetes/server/bin/{kubelet,kube-proxy} root@${node_ip}:/opt/k8s/bin/
    ssh root@${node_ip} "chmod +x /opt/k8s/bin/*"
  done

在kubernetes 1.15版起kubelet需要删除下面的参数，因为新版本中这个参数已经过时。我们只需要在kubelet启动脚步中删除即可参考https://github.com/microsoft/SDN/issues/379

#需要在kubelet启动文件删除相关参数
allow-privileged=true

#这里需要在每台node节点执行！
sed -i '/allow-privileged/d'  /etc/systemd/system/kubelet.service
systemctl daemon-reload && systemctl start kubelet

执行完命令后可以查看一下kubelet日志，检查是否有报错
journalctl -fu kubelet

kubelet成功启动后通过kubectl get node所有节点的版本已经变为1.15.11 这里的过程可能比较慢，耐心等待一会就好了。如果长时间处于NotReady就需要查看一下kubelet日志

[root@k8s-01 ~]# kubectl get node
NAME     STATUS   ROLES    AGE     VERSION
k8s-01   Ready       3d10h   v1.15.11
k8s-02   Ready       3d10h   v1.15.11
k8s-03   Ready       3d10h   v1.15.11
k8s-04   Ready       3d10h   v1.15.11
k8s-05   Ready       3d10h   v1.15.11

目前还没有结束，我们还需要启动kube-proxy

for i in k8s-01 k8s-02 k8s-03 k8s-04 k8s-05
do
  ssh root@$i systemctl start kube-proxy
done

这里很多的命令都可以通过之前安装k8s的脚本进行获取，比如查看一下kube-proxy状态

source /opt/k8s/bin/environment.sh
for node_ip in ${NODE_IPS[@]}
  do
    echo ">>> ${node_ip}"
    ssh root@${node_ip} "systemctl status kube-proxy|grep Active"
  done

接下来我们可以验证一下集群的状态

[root@k8s-01 ~]# kubectl cluster-info 
Kubernetes master is running at https://192.168.31.105:8443
CoreDNS is running at https://192.168.31.105:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@k8s-01 ~]# kubectl get node
NAME     STATUS   ROLES    AGE     VERSION
k8s-01   Ready       3d11h   v1.15.11
k8s-02   Ready       3d11h   v1.15.11
k8s-03   Ready       3d11h   v1.15.11
k8s-04   Ready       3d11h   v1.15.11
k8s-05   Ready       3d11h   v1.15.11
[root@k8s-01 ~]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
etcd-0               Healthy   {"health":"true"}   
etcd-2               Healthy   {"health":"true"}   
etcd-1               Healthy   {"health":"true"}   
scheduler            Healthy   ok                  
controller-manager   Healthy   ok

由于之前我这里安装了coredns，所以我在检查一下coredns是否正常

#创建测试yaml

cat<
相关文章：
Kubernetes 1.14 二进制集群安装 
Kubenetes 1.13.5 集群二进制安装 
Kuerbernetes 1.11 集群二进制安装 
CentOS 7 ETCD集群配置大全

Kubernetes 二进制升级

Kubernetes 1.14 二进制集群安装

升级kubectl

升级master组件

升级node组件

相关文章：

Rego 不好用？用 Pipy 实现 OPA

微服务断路器模式实现：Istio vs Hystrix

服务网格之路

Axios ajax

Telnet、SSH 原理