运维首页 > 应用运维 > ingressnginx的rewrite与canary

ingressnginx的rewrite与canary

2023年 7月 15日 50.4k 0

ingress-nginx的官网提供了更多的一些配置信息，包括url重写，金丝雀，尽管金丝雀支持并不完美，ingress-nginx仍然是最受欢迎的ingress之一。在上一篇中，我介绍了ingress-nginx应用常见的两种方式,并且采用的最新的版本，早期有非常陈旧的版本在使用。鉴于此，随后打算将ingress-nginx重新理一遍，于是就有了这篇，后续可能还会有

ingress-nginx本身只是做一个声明，从哪里来到哪里去而已，并不会做一些流量转发，而核心是annotations的class是可以借助作一些操作的，比如修改城Traefik或者自己定制

创建一个deployment的pod，我们至少需要指定标签如下

  selector:
    matchLabels:
      app: linuxea_app
      version: v0.1.32
  template:
    metadata:
      labels:
        app: linuxea_app
        version: v0.1.32

而后在service关联

  selector:
    app: linuxea_app
    version: v0.1.32

并配置一个ingress，name: myapp必须是这个service的name , 且必须在同一个名称空间，如下

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test
  namespace: default
spec:
  tls:
  - hosts:
    - linuxea.test.com
    secretName: nginx-ingress-secret
  ingressClassName: nginx
  rules:
  - host: linuxea.test.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp
            port:
              number: 80
---
apiVersion: v1
kind: Service
metadata:
  name: myapp
  namespace: default
spec:
  selector:
    app: linuxea_app
    version: v0.1.32
  ports:
  - name: http
    targetPort: 80
    port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: dpment-linuxea
  namespace: default
spec:
  replicas: 7
  selector:
    matchLabels:
      app: linuxea_app
      version: v0.1.32
  template:
    metadata:
      labels:
        app: linuxea_app
        version: v0.1.32
    spec:
      containers:
      - name: nginx-a
        image: marksugar/nginx:1.14.b
        ports:
        - name: http
          containerPort: 80

这样就完成了一个最简单不过的ingress-nginx的域名配置，当然也可以配置一个404的页面之类的

而整个过程大致如下

image-20220322214835499.png

请求到达LB后被分发到ingress-controller，controller会一直关注ingress对象，匹配到对应的信息后将请求转发到其中的某一个pod之上，而service对后端的pod做发现和关联。意思就是ingress-nginx不是直接到service，而是只从service中获取pod的信息，service仍然负责发现后端的pod状态，而请求是通过ingress通过ed到pod的

url重写

ingress-nginx大致如下，除此之外，我们可以对annotation做一些配置，较为常见的rewrite功能

在实际中，访问的url可能如下

linuxea.test.com/qsv1
linuxea.test.com/qsv2
linuxea.test.com/qsv3

诸如此类，而要进行这种跳转，需要前端代码支持，或者配置rewrite进行转发，如下

Name Description Values
nginx.ingress.kubernetes.io/rewrite-target Target URI where the traffic must be redirected string
nginx.ingress.kubernetes.io/ssl-redirect Indicates if the location section is only accessible via SSL (defaults to True when Ingress contains a Certificate) bool
nginx.ingress.kubernetes.io/force-ssl-redirect Forces the redirection to HTTPS even if the Ingress is not TLS Enabled bool
nginx.ingress.kubernetes.io/app-root Defines the Application Root that the Controller must redirect if it's in / context string
nginx.ingress.kubernetes.io/use-regex Indicates if the paths defined on an Ingress use regular expressions bool 
Captured groups are saved in numbered placeholders, chronologically, in the form $1, $2 ... $n. These placeholders can be used as parameters in the rewrite-target annotation.

nginx.ingress.kubernetes.io/rewrite-target 将请求转发到目标

比如现在要将/app转发到/app/modiy，那么就可以如下正则表达

/app(/|$)(.*)

并且rewrite-target，的值是一个$2占位符

  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
...
      paths:
      - path: /app(/|$)(.*)

而这种方式i还有一个问题，就是你的js代码很有可能是绝对路径的，因此你不能够打开js，js会404 。 要么修改为相对路径，要么就需要重新配置一个重定向

假设你的jss样式在/style下，还可能有图片是在image下以及js的路径，和其他增删改查的页面，现在的跳转后央视404，可以使用configuration-snippet重写

  • configuration-snippet
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/app-root: /linuxea.html
    nginx.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^/style/(.*)$ /app/style/$1 redirect;
      rewrite ^/image/(.*)$ /app/image/$1 redirect;
      rewrite ^/javascripts/(.*)$ /app/javascripts/$1 redirect;
      rewrite ^/modiy/(.*)$ /app/modiy/$1 redirect;
      rewrite ^/create/(.*)$ /app/create/$1 redirect;      
      rewrite ^/delete/(.*)$ /app/delete/$1 redirect;

|表示换行，而后rewrite以/style/路径下的所有跳转到/app/style/下，完成对style添加前缀/app

  • app-root

如果此时我们希望访问的根目录不是默认的，可以使用app-root来进行跳转，比如跳转到linuxea.html

如果是一个目录，就可以写一个路径，比如/app/

  annotations:
    nginx.ingress.kubernetes.io/app-root: /linuxea.html
[root@Node-172_16_100_50 ~/ingress]# kubectl apply -f ingress.yaml
ingress.networking.k8s.io/test configured

image-20220322223245437.png

现在就完成了自动跳转

basic auth认证

在nginx里面是可以配置basic auth认证的，非常简单的一个配置，在ingress-nginx中也是可以的

我们可以进行yum安装一个httpd的应用，或者在搜索引擎搜索一个在线 htpasswd 生成器来生成一个

用户mark，密码linuxea.com

  • yum install httpd -y
# htpasswd -c auth mark
New password: 
Re-type new password: 
Adding password for user mark
  • 在线生成即可
# cat auth1
mark:$apr1$69ocxsQr$omgzB53m59LeCVxlOAsTr/

创建一个secret，将这个文件配置即可

kubectl create secret generic bauth --from-file=auth1
# kubectl create secret generic bauth --from-file=auth1
secret/bauth created
# kubectl get secret bauth
NAME    TYPE     DATA   AGE
bauth   Opaque   1      21s
# kubectl describe secret bauth
Name:         bauth
Namespace:    default
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
auth1:  43 bytes

而后添加到ingress-nginx中，如下

  annotations:
    ...
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: auth-1
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication failed, please try again'

auth-secret是引入刚创建的bauth,而auth-type指定了类型，如下

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/app-root: /linuxea.html
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: auth-1
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication failed, please try again'
spec:
  tls:
  - hosts:
    - linuxea.test.com
    secretName: nginx-ingress-secret
  ingressClassName: nginx
  rules:
  - host: linuxea.test.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp
            port:
              number: 80

执行一下

# kubectl apply -f ingress.yaml 
ingress.networking.k8s.io/test configured

如下

image-20220322230038421.png

灰度

我们通常使用最多的滚动更新，蓝绿，灰度，而ingress-ngiinx是通过annotations配置来实现的，能满足金丝雀，蓝绿、ab测试

缺少描述 部分

此前我们配置了一个pod和一个service，要配置金丝雀那就需要在配置一组，而后我们在ingress中使用annotations来进行调用其他的一些class来完成一些操作

配置nginx:v1.14.a

apiVersion: apps/v1
kind: Deployment
metadata:
  name: testv1
  labels:
    app: testv1
spec:
  replicas: 5
  selector:
    matchLabels:
      app: testv1
  template:
    metadata:
      labels:
        app: testv1
    spec:
      containers:
      - name: testv1
        image: marksugar/nginx:1.14.a
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: testv1-service
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: testv1

testv2

apiVersion: apps/v1
kind: Deployment
metadata:
  name: testv2
  labels:
    app: testv2
spec:
  replicas: 5
  selector:
    matchLabels:
      app: testv2
  template:
    metadata:
      labels:
        app: testv2
    spec:
      containers:
      - name: testv2
        image: marksugar/nginx:1.14.b
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: testv2-service
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: testv2

apply

# kubectl apply -f testv1.yaml
# kubectl apply -f testv2.yaml

可以看到现在已经有两组

# kubectl  get pod
NAME                      READY   STATUS    RESTARTS   AGE
testv1-9c974bd5d-c46dh                   1/1     Running   0          19s
testv1-9c974bd5d-j7fzn                   1/1     Running   0          19s
testv1-9c974bd5d-qp4tv                   1/1     Running   0          19s
testv1-9c974bd5d-thx4r                   1/1     Running   0          19s
testv1-9c974bd5d-x9rpf                   1/1     Running   0          19s
testv2-5767685995-f8z5s                  1/1     Running   0          6s
testv2-5767685995-htm74                  1/1     Running   0          6s
testv2-5767685995-k8sdv                  1/1     Running   0          6s
testv2-5767685995-mjd6c                  1/1     Running   0          6s
testv2-5767685995-prhld                  1/1     Running   0          6s

给testv1配置一个 ingress-v1.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: testv1
  namespace: default
spec:
  ingressClassName: nginx
  rules:
  - host: test.mark.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: testv1-service
            port:
              number: 80
# kubectl apply -f ingress-v1.yaml

而后我们查看的版本信息

# for i in $(seq 1 10);do curl -s test.mark.com/linuxea.html ;done
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-x9rpf.com ▍ 9cbb453617c73 ▍version number 1.0
linuxea-testv1-9c974bd5d-c46dh.com ▍ 4c0e80c7d9a34 ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-x9rpf.com ▍ 9cbb453617c73 ▍version number 1.0
linuxea-testv1-9c974bd5d-thx4r.com ▍ b9e074d68c3c7 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0

canary

而后我们配置canary

    nginx.ingress.kuberentes.io/canary: "true"  # 开启灰度发布机制，首先启用canary
    nginx.ingress.kuberentes.io/canary-weight: "30" # 分配30%的流量到当前的canary版本

如下

给testv2配置一个 ingress-v2.yaml 并配置canary权重

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: testv2
  namespace: default
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
spec:
  ingressClassName: nginx
  rules:
  - host: test.mark.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: testv2-service
            port:
              number: 80

此时由于版本问题你或许会发现 有一个问题

Error from server (BadRequest): error when creating "ingress-1.yaml": admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "linuxea.test.com" and path "/" is already defined in ingress default/test

而这个问题的根源在于没有验证webhook忽略具有不同的ingressclass的入口

controller.admissionWebhooks.enabled=false

并且在1.1.2修复

我们安装1.1.2的ingress-nginx

docker pull k8s.gcr.io/ingress-nginx/controller:v1.1.2@sha256:28b11ce69e57843de44e3db6413e98d09de0f6688e33d4bd384002a44f78405c

找到一个aliyun的

docker pull registry.cn-shanghai.aliyuncs.com/wanfei/ingress-nginx-controller:v1.1.2
docker pull registry.cn-shanghai.aliyuncs.com/wanfei/kube-webhook-certgen:v1.1.1
docker pull registry.cn-shanghai.aliyuncs.com/wanfei/defaultbackend-amd64:1.5

修改ingress-nginx的deployment.yaml而后在配置下

应用声明式的文件

# kubectl apply -f ingress-v2.yaml 
#  kubectl get ingress
NAME      CLASS   HOSTS              ADDRESS         PORTS   AGE
linuxea   nginx   linuxea.test.com   172.16.100.50   80      22h
testv1    nginx   test.mark.com      172.16.100.50   80      3m23s
testv2    nginx   test.mark.com      172.16.100.50   80      70s
for i in $(seq 1 10);do curl -s linuxea.test.com ;done
# for i in $(seq 1 10);do curl -s test.mark.com/linuxea.html ;done
linuxea-testv1-9c974bd5d-thx4r.com ▍ b9e074d68c3c7 ▍version number 1.0
linuxea-testv1-9c974bd5d-c46dh.com ▍ 4c0e80c7d9a34 ▍version number 1.0
linuxea-testv2-5767685995-mjd6c.com ▍ 1fa571f0e1e0e ▍version number 2.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-thx4r.com ▍ b9e074d68c3c7 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-x9rpf.com ▍ 9cbb453617c73 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0

这里的比例是大致的一个算法，而并不是固定的

此时可以将weight配置成0撤销更新

 nginx.ingress.kubernetes.io/canary-weight: "0"

或者将weight配置成100完成更新

 nginx.ingress.kubernetes.io/canary-weight: "100"

参考：

validating webhook should ignore ingresses with a different ingressclass

validating webhook should ignore ingresses with a different ingressclass

slack讨论

Error: admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "xyz" and path "/" is already defined in ingress xxx #821kubernetes Ingress Controller (15)kubernetes Ingress nginx http以及7层https配置 (17)kubernetes Ingress nginx配置 (16)

LOVEHL^ˇ^
应用运维
0 0

相关文章

LeaferJS 1.0 重磅发布：强悍的前端 Canvas 渲染引擎
10分钟搞定支持通配符的永久有效免费HTTPS证书
300 多个 Microsoft Excel 快捷方式
一步步配置基于kubeadmin的kubevip高可用
istio全链路传递cookie和header灰度
REST Web 服务版本控制

发布评论

LOVEHL^ˇ^

这个人很懒，什么都没有留下～

文章8438 评论0