运维首页 > 云运维 > kubernetes calico IPV6支持

kubernetes calico IPV6支持

2023年 7月 9日 43.9k 0

1.说明

实际上IPV6和IPV4在在配置上没有太大差异，本次只在配置上做相关说明。由于公司的云环境还不支持IPV6，本次主要在虚拟机上完成。

主机规划

名称	IPV4	IPV6
master	192.168.6.110	fd00::20c:29ff:fe9f:52be
node2	192.168.6.103	fd00::39df:8f1b:e228:d42
node3	192.168.6.113	fd00::20c:29ff:fead:d381

网络规划

名称	协议
service-cluster-ip-range	fd03::/120
service-node-port-range	30000-32767
cluster-cidr	fd05::/120
cluster-dns	fd05::2
node-cidr-mask-size	121

2.环境准备

高版本的VMware开启IPV6支持，同时设置IPV6的网络地址范围。

kubernetes calico IPV6支持-1

3.组件配置

1.对虚拟机的配置（三台操作）

#增加配置
[root@master ~]# vim /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
net.ipv6.conf.all.forwarding=1
#使之生效
[root@master ~]# sysctl -p
[root@master ~]# vim /etc/sysconfig/network
#添加
NETWORKING_IPV6=yes
[root@master ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
#添加
IPV6INIT=yes
IPV6_AUTOCONF=yes
[root@master ~]# reboot

证书源文件中的配置。此处不会过多介绍安装。关键介绍下配置部分，安装可参考Kubernetes安装.

apiserver-csr.json

{
  "CN": "kube-apiserver",
  "hosts": [
      "10.254.0.1",
      "192.168.6.110",
      "192.168.6.112",
      "192.168.6.130",
      "192.168.6.113",
      // 主机
       "fd00::20c:29ff:fe9f:52be",
       "fd00::39df:8f1b:e228:d42",
       "fd00::20c:29ff:fead:d381",
       "127.0.0.1",
       "::1",
        "fd03::1",
        "fd05::1",
        "localhost",
        "kubernetes",
        "kubernetes.default",
        "kubernetes.default.svc",
        "kubernetes.default.svc.cluster",
        "kubernetes.default.svc.cluster.local"
        ],
        "key": {
                "algo": "rsa",
                "size": 2048
        },
        "names": [{
                "C": "CN",
                "ST": "NanJing",
                "L": "NanJing",
                "O": "Kubernetes",
                "OU": "Kubernetes-manual"
        }]
}

etcd-csr.json 此处为了省事，我把IPV4和IPV6全部加上了。

{
  "CN": "etcd",
  "hosts": [
    "192.168.6.110",
    "192.168.6.112",
    "192.168.6.130",
    "192.168.6.113",
    "fd00::20c:29ff:fe9f:52be",
    "fd00::39df:8f1b:e228:d42",
    "fd00::20c:29ff:fead:d381",
    "127.0.0.1",
    "::1"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "NanJing",
      "L": "NanJing",
      "O": "Kubernetes",
      "OU": "Kubernetes-manual"
    }
  ]
}

3.Etcd的配置实际上把IPV4换成IPV6即可，注意写法。

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
#User=etcd
ExecStart=/data/cloud/etcd/etcd \
--name=node1 \
--heartbeat-interval=500 \
--election-timeout=5000 \
--cert-file=/data/cloud/pki/etcd.pem \
--key-file=/data/cloud/pki/etcd-key.pem \
--trusted-ca-file=/data/cloud/pki/ca.pem \
--peer-cert-file=/data/cloud/pki/etcd.pem \
--peer-key-file=/data/cloud/pki/etcd-key.pem \
--peer-trusted-ca-file=/data/cloud/pki/ca.pem \
--initial-advertise-peer-urls=https://[fd00::20c:29ff:fe9f:52be]:2380 \
--listen-peer-urls=https://[fd00::20c:29ff:fe9f:52be]:2380 \
--listen-client-urls=https://[fd00::20c:29ff:fe9f:52be]:2379,http://[::1]:2379 \
--advertise-client-urls=https://[fd00::20c:29ff:fe9f:52be]:2379 \
--initial-cluster-token=kubernetes \
--initial-cluster=node1=https://[fd00::20c:29ff:fe9f:52be]:2380,node2=https://[fd00::39df:8f1b:e228:d42]:2380,node3=https://[fd00::20c:29ff:fead:d381]:2380 \
--initial-cluster-state=new \
--data-dir=/data/cloud/work/etcd

Restart=on-failure
RestartSec=10
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

5.apiserver的关键配置

 --bind-address=:: \ 相当于IPV4中的0.0.0.0
    --secure-port=6443 \
    --insecure-port=0 \ 禁用非安全端口
    --advertise-address=fd00::20c:29ff:fe9f:52be \
    --service-cluster-ip-range=fd03::/120 \ 设置serviceip的范围
    --service-node-port-range=30000-32767 \
    --etcd-servers=https://[fd00::20c:29ff:fe9f:52be]:2379,https://[fd00::20c:29ff:fe83:39c3]:2379,https://[fd00::20c:29ff:fead:d381]:2379 \
//其他部分略......

6.controller-manager的关键配置

ExecStart=/data/cloud/kubernetes/bin/kube-controller-manager \
  --bind-address=:: \
  --allocate-node-cidrs=true \
  --cluster-cidr=fd05::/120 \
  --node-cidr-mask-size=121 \ 此处除以一点要比上面的120大

6.scheduler的关键配置

ExecStart=/data/cloud/kubernetes/bin/kube-scheduler \
  --bind-address=:: \
  --leader-elect=true \
  --logtostderr=false \
  --kubeconfig=/data/cloud/pki/scheduler.conf \
  --log-dir=/data/cloud/work/kubernetes/kube-scheduler \
  --v=2

7.kubelet 的关键配置

ExecStart=/data/cloud/kubernetes/bin/kubelet \
  --fail-swap-on=false \
  --address=:: \
  --healthz-bind-address=:: \
  --hostname-override=node2 \
  --node-ip=fd00::39df:8f1b:e228:d42 \ //此处一定要加上，不然默认注册的是IPV4
  --pod-infra-container-image=k8s.gcr.io/pause:3.1 \
  --network-plugin=cni  --cni-bin-dir=/opt/cni/bin \
  --kubeconfig=/data/cloud/pki/kubelet.conf \
  --bootstrap-kubeconfig=/data/cloud/pki/bootstrap.conf \
  --pod-manifest-path=/data/cloud/kubernetes/manifests \
  --allow-privileged=true \
  --cluster-dns=fd05::2 \ //设置DNS的地址

8.kubelet 的关键配置

ExecStart=/data/cloud/kubernetes/bin/kube-proxy \
 --bind-address=:: \
 --hostname-override=node2 \
 --cluster-cidr=fd05::/120 \
 --kubeconfig=/data/cloud/pki/proxy.conf \
 --logtostderr=true \
 --log-dir=/data/cloudwork/kubernetes/kube-proxy \
 --v=2

9.docker的配置 /etc/docker/daemon.json

{
"insecure-registry":["0.0.0.0/0"],
"ipv6": true,
"fixed-cidr-v6": "2001:db8:1::/64",
"data-root": "/data/cloud/work/docker", //我一般会调整其默认的工作目录
"host":["unix:///var/run/docker.sock","tcp://:::2375"],
"log-level":"debug"
}

4.启动组件进行验证

集群整体状况
为了方便操作做了个别名
alias kubectl=’kubectl –kubeconfig=/data/cloud/pki/admin.conf’

[root@node1 system]# kubectl get cs
NAME                 STATUS      MESSAGE                                                                                                                   ERROR
controller-manager   Healthy     ok                                                                                                                        
scheduler            Healthy     ok                                                                                                                        
etcd-0               Healthy     {"health":"true"}                                                                                                         
etcd-2               Healthy     {"health":"true"}                                                                                                         
etcd-1               Healthy     {"health":"true"}

节点状况,此处可以看到INTERNAL-IP注册上来的是IPV6地址

[root@node1 system]# kubectl get no -owide  
NAME    STATUS   ROLES    AGE   VERSION   INTERNAL-IP                EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION              CONTAINER-RUNTIME
node2   Ready       36m   v1.13.0   fd00::39df:8f1b:e228:d42           CentOS Linux 7 (Core)   3.10.0-957.el7.x86_64       docker://18.9.3
node3   Ready       51m   v1.13.0   fd00::20c:29ff:fead:d381           CentOS Linux 7 (Core)   3.10.0-957.5.1.el7.x86_64   docker://18.9.2

kubernetes 分配的service ip

[root@node1 system]# kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   fd03::1              443/TCP   3h34m

calico的配置

//指定证书位置
kubeconfig_filepath: "/data/cloud/pki/admin.conf"

calico部署

[root@node1 calico]# kubectl apply -f calico.yaml
[root@node1 yaml]# kubectl get pod -n kube-system -owide 
NAME                READY   STATUS    RESTARTS   AGE    IP                         NODE    NOMINATED NODE   READINESS GATES
calico-node-mwkj5   1/1     Running   0          87m    fd00::39df:8f1b:e228:d42   node2              
calico-node-vjhpb   1/1     Running   0          102m   fd00::20c:29ff:fead:d381   node3

测试

[root@node1 yaml]# kubectl run tomcat  --image=tomcat:8.0  --replicas=2 --port=8080
[root@node1 yaml]# kubectl get pod -owide 
NAME                      READY   STATUS    RESTARTS   AGE   IP         NODE    NOMINATED NODE   READINESS GATES
tomcat-79d98465c6-jqvgp   1/1     Running   0          17s   fd05::b    node3              
tomcat-79d98465c6-n4rgh   1/1     Running   0          17s   fd05::86   node2              
//node2直接访问容器
[root@node2 images]# curl -6g  [fd05::86]:8080





    
        
        Apache Tomcat/8.5.38
        
        
        
    

······
//node3直接访问容器
[root@node3 cloud]# curl -6g  [fd05::b]:8080





    
        
        Apache Tomcat/8.5.38
        
        
        
    
······
//暴露端口
[root@node1 calico]# kubectl expose deployment tomcat  --port=8080 --target-port=8080 --type=NodePort 
service/myip exposed
[root@node1 yaml]# kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   fd03::1              443/TCP          6h38m
tomcat       NodePort    fd03::46             8080:31900/TCP   21m

通过serviceIP+端口访问(8080),因为master未安装kube-proxy，在node2和node3上访问,为了省事此处截图。

kubernetes calico IPV6支持-4

通过node+端口访问(31900)

kubernetes calico IPV6支持-5

法医

ipv 云运维

0 0

《k8s1.13版本源码分析》调度优选

如何评估Kubernetes持久化存储方案

KubeSphere 部署向量数据库 Milvus 实战指南

探索 Kubernetes 持久化存储之 Longhorn 初窥门径

征服 Docker 镜像访问限制！KubeSphere v3.4.1 成功部署全攻略

那些年在 Terraform 上吃到的糖和踩过的坑

无需 Kubernetes 测试 Kubernetes 网络实现

Kubernetes v1.31 中的移除和主要变更

发布评论取消回复

要发表评论，您必须先登录。

法医

这个人很懒，什么都没有留下～

文章8313 评论0