ingress-nginx的官网提供了更多的一些配置信息,包括url重写,金丝雀,尽管金丝雀支持并不完美,ingress-nginx仍然是最受欢迎的ingress之一。在上一篇中,我介绍了ingress-nginx应用常见的两种方式,并且采用的最新的版本,早期有非常陈旧的版本在使用。鉴于此,随后打算将ingress-nginx重新理一遍,于是就有了这篇,后续可能还会有
ingress-nginx本身只是做一个声明,从哪里来到哪里去而已,并不会做一些流量转发,而核心是annotations的class是可以借助作一些操作的,比如修改城Traefik或者自己定制
创建一个deployment的pod,我们至少需要指定标签如下
selector:
matchLabels:
app: linuxea_app
version: v0.1.32
template:
metadata:
labels:
app: linuxea_app
version: v0.1.32
而后在service关联
selector:
app: linuxea_app
version: v0.1.32
并配置一个ingress,name: myapp必须是这个service的name , 且必须在同一个名称空间,如下
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test
namespace: default
spec:
tls:
- hosts:
- linuxea.test.com
secretName: nginx-ingress-secret
ingressClassName: nginx
rules:
- host: linuxea.test.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp
port:
number: 80
---
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: default
spec:
selector:
app: linuxea_app
version: v0.1.32
ports:
- name: http
targetPort: 80
port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: dpment-linuxea
namespace: default
spec:
replicas: 7
selector:
matchLabels:
app: linuxea_app
version: v0.1.32
template:
metadata:
labels:
app: linuxea_app
version: v0.1.32
spec:
containers:
- name: nginx-a
image: marksugar/nginx:1.14.b
ports:
- name: http
containerPort: 80
这样就完成了一个最简单不过的ingress-nginx的域名配置,当然也可以配置一个404的页面之类的
而整个过程大致如下
请求到达LB后被分发到ingress-controller,controller会一直关注ingress对象,匹配到对应的信息后将请求转发到其中的某一个pod之上,而service对后端的pod做发现和关联。意思就是ingress-nginx不是直接到service,而是只从service中获取pod的信息,service仍然负责发现后端的pod状态,而请求是通过ingress通过ed到pod的
url重写
ingress-nginx大致如下,除此之外,我们可以对annotation做一些配置,较为常见的rewrite功能
在实际中,访问的url可能如下
linuxea.test.com/qsv1
linuxea.test.com/qsv2
linuxea.test.com/qsv3
诸如此类,而要进行这种跳转,需要前端代码支持,或者配置rewrite进行转发,如下
Name | Description | Values |
---|---|---|
nginx.ingress.kubernetes.io/rewrite-target | Target URI where the traffic must be redirected | string |
nginx.ingress.kubernetes.io/ssl-redirect | Indicates if the location section is only accessible via SSL (defaults to True when Ingress contains a Certificate) | bool |
nginx.ingress.kubernetes.io/force-ssl-redirect | Forces the redirection to HTTPS even if the Ingress is not TLS Enabled | bool |
nginx.ingress.kubernetes.io/app-root | Defines the Application Root that the Controller must redirect if it's in / context |
string |
nginx.ingress.kubernetes.io/use-regex | Indicates if the paths defined on an Ingress use regular expressions | bool |
Captured groups are saved in numbered placeholders, chronologically, in the form $1, $2 ... $n. These placeholders can be used as parameters in the rewrite-target annotation.
nginx.ingress.kubernetes.io/rewrite-target
将请求转发到目标
比如现在要将/app转发到/app/modiy,那么就可以如下正则表达
/app(/|$)(.*)
并且rewrite-target,的值是一个$2占位符
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
...
paths:
- path: /app(/|$)(.*)
而这种方式i还有一个问题,就是你的js代码很有可能是绝对路径的,因此你不能够打开js,js会404 。 要么修改为相对路径,要么就需要重新配置一个重定向
假设你的jss样式在/style下,还可能有图片是在image下以及js的路径,和其他增删改查的页面,现在的跳转后央视404,可以使用configuration-snippet重写
- configuration-snippet
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/app-root: /linuxea.html
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite ^/style/(.*)$ /app/style/$1 redirect;
rewrite ^/image/(.*)$ /app/image/$1 redirect;
rewrite ^/javascripts/(.*)$ /app/javascripts/$1 redirect;
rewrite ^/modiy/(.*)$ /app/modiy/$1 redirect;
rewrite ^/create/(.*)$ /app/create/$1 redirect;
rewrite ^/delete/(.*)$ /app/delete/$1 redirect;
|表示换行,而后rewrite以/style/路径下的所有跳转到/app/style/下,完成对style添加前缀/app
- app-root
如果此时我们希望访问的根目录不是默认的,可以使用app-root来进行跳转,比如跳转到linuxea.html
如果是一个目录,就可以写一个路径,比如/app/
annotations:
nginx.ingress.kubernetes.io/app-root: /linuxea.html
[root@Node-172_16_100_50 ~/ingress]# kubectl apply -f ingress.yaml
ingress.networking.k8s.io/test configured
现在就完成了自动跳转
basic auth认证
在nginx里面是可以配置basic auth认证的,非常简单的一个配置,在ingress-nginx中也是可以的
我们可以进行yum安装一个httpd的应用,或者在搜索引擎搜索一个在线 htpasswd 生成器来生成一个
用户mark,密码linuxea.com
- yum install httpd -y
# htpasswd -c auth mark
New password:
Re-type new password:
Adding password for user mark
- 在线生成即可
# cat auth1
mark:$apr1$69ocxsQr$omgzB53m59LeCVxlOAsTr/
创建一个secret,将这个文件配置即可
kubectl create secret generic bauth --from-file=auth1
# kubectl create secret generic bauth --from-file=auth1
secret/bauth created
# kubectl get secret bauth
NAME TYPE DATA AGE
bauth Opaque 1 21s
# kubectl describe secret bauth
Name: bauth
Namespace: default
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
auth1: 43 bytes
而后添加到ingress-nginx中,如下
annotations:
...
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: auth-1
nginx.ingress.kubernetes.io/auth-realm: 'Authentication failed, please try again'
auth-secret是引入刚创建的bauth,而auth-type指定了类型,如下
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: test
namespace: default
annotations:
nginx.ingress.kubernetes.io/app-root: /linuxea.html
nginx.ingress.kubernetes.io/auth-type: basic
nginx.ingress.kubernetes.io/auth-secret: auth-1
nginx.ingress.kubernetes.io/auth-realm: 'Authentication failed, please try again'
spec:
tls:
- hosts:
- linuxea.test.com
secretName: nginx-ingress-secret
ingressClassName: nginx
rules:
- host: linuxea.test.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp
port:
number: 80
执行一下
# kubectl apply -f ingress.yaml
ingress.networking.k8s.io/test configured
如下
灰度
我们通常使用最多的滚动更新,蓝绿,灰度,而ingress-ngiinx是通过annotations配置来实现的,能满足金丝雀,蓝绿、ab测试
缺少描述 部分
此前我们配置了一个pod和一个service,要配置金丝雀那就需要在配置一组,而后我们在ingress中使用annotations来进行调用其他的一些class来完成一些操作
配置nginx:v1.14.a
apiVersion: apps/v1
kind: Deployment
metadata:
name: testv1
labels:
app: testv1
spec:
replicas: 5
selector:
matchLabels:
app: testv1
template:
metadata:
labels:
app: testv1
spec:
containers:
- name: testv1
image: marksugar/nginx:1.14.a
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: testv1-service
spec:
ports:
- port: 80
targetPort: 80
selector:
app: testv1
testv2
apiVersion: apps/v1
kind: Deployment
metadata:
name: testv2
labels:
app: testv2
spec:
replicas: 5
selector:
matchLabels:
app: testv2
template:
metadata:
labels:
app: testv2
spec:
containers:
- name: testv2
image: marksugar/nginx:1.14.b
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: testv2-service
spec:
ports:
- port: 80
targetPort: 80
selector:
app: testv2
apply
# kubectl apply -f testv1.yaml
# kubectl apply -f testv2.yaml
可以看到现在已经有两组
# kubectl get pod
NAME READY STATUS RESTARTS AGE
testv1-9c974bd5d-c46dh 1/1 Running 0 19s
testv1-9c974bd5d-j7fzn 1/1 Running 0 19s
testv1-9c974bd5d-qp4tv 1/1 Running 0 19s
testv1-9c974bd5d-thx4r 1/1 Running 0 19s
testv1-9c974bd5d-x9rpf 1/1 Running 0 19s
testv2-5767685995-f8z5s 1/1 Running 0 6s
testv2-5767685995-htm74 1/1 Running 0 6s
testv2-5767685995-k8sdv 1/1 Running 0 6s
testv2-5767685995-mjd6c 1/1 Running 0 6s
testv2-5767685995-prhld 1/1 Running 0 6s
给testv1配置一个 ingress-v1.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: testv1
namespace: default
spec:
ingressClassName: nginx
rules:
- host: test.mark.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: testv1-service
port:
number: 80
# kubectl apply -f ingress-v1.yaml
而后我们查看的版本信息
# for i in $(seq 1 10);do curl -s test.mark.com/linuxea.html ;done
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-x9rpf.com ▍ 9cbb453617c73 ▍version number 1.0
linuxea-testv1-9c974bd5d-c46dh.com ▍ 4c0e80c7d9a34 ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-x9rpf.com ▍ 9cbb453617c73 ▍version number 1.0
linuxea-testv1-9c974bd5d-thx4r.com ▍ b9e074d68c3c7 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
canary
而后我们配置canary
nginx.ingress.kuberentes.io/canary: "true" # 开启灰度发布机制,首先启用canary
nginx.ingress.kuberentes.io/canary-weight: "30" # 分配30%的流量到当前的canary版本
如下
给testv2配置一个 ingress-v2.yaml 并配置canary权重
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: testv2
namespace: default
annotations:
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "30"
spec:
ingressClassName: nginx
rules:
- host: test.mark.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: testv2-service
port:
number: 80
此时由于版本问题你或许会发现 有一个问题
Error from server (BadRequest): error when creating "ingress-1.yaml": admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "linuxea.test.com" and path "/" is already defined in ingress default/test
而这个问题的根源在于没有验证webhook忽略具有不同的ingressclass的入口
controller.admissionWebhooks.enabled=false
并且在1.1.2修复
我们安装1.1.2的ingress-nginx
docker pull k8s.gcr.io/ingress-nginx/controller:v1.1.2@sha256:28b11ce69e57843de44e3db6413e98d09de0f6688e33d4bd384002a44f78405c
找到一个aliyun的
docker pull registry.cn-shanghai.aliyuncs.com/wanfei/ingress-nginx-controller:v1.1.2 docker pull registry.cn-shanghai.aliyuncs.com/wanfei/kube-webhook-certgen:v1.1.1 docker pull registry.cn-shanghai.aliyuncs.com/wanfei/defaultbackend-amd64:1.5
修改ingress-nginx的deployment.yaml而后在配置下
应用声明式的文件
# kubectl apply -f ingress-v2.yaml
# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
linuxea nginx linuxea.test.com 172.16.100.50 80 22h
testv1 nginx test.mark.com 172.16.100.50 80 3m23s
testv2 nginx test.mark.com 172.16.100.50 80 70s
for i in $(seq 1 10);do curl -s linuxea.test.com ;done
# for i in $(seq 1 10);do curl -s test.mark.com/linuxea.html ;done
linuxea-testv1-9c974bd5d-thx4r.com ▍ b9e074d68c3c7 ▍version number 1.0
linuxea-testv1-9c974bd5d-c46dh.com ▍ 4c0e80c7d9a34 ▍version number 1.0
linuxea-testv2-5767685995-mjd6c.com ▍ 1fa571f0e1e0e ▍version number 2.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-thx4r.com ▍ b9e074d68c3c7 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-qp4tv.com ▍ f6f9a8a43d4ee ▍version number 1.0
linuxea-testv1-9c974bd5d-x9rpf.com ▍ 9cbb453617c73 ▍version number 1.0
linuxea-testv1-9c974bd5d-j7fzn.com ▍ d471888034671 ▍version number 1.0
这里的比例是大致的一个算法,而并不是固定的
此时可以将weight配置成0撤销更新
nginx.ingress.kubernetes.io/canary-weight: "0"
或者将weight配置成100完成更新
nginx.ingress.kubernetes.io/canary-weight: "100"
参考:
validating webhook should ignore ingresses with a different ingressclass
validating webhook should ignore ingresses with a different ingressclass
slack讨论
Error: admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "xyz" and path "/" is already defined in ingress xxx #821kubernetes Ingress Controller (15)kubernetes Ingress nginx http以及7层https配置 (17)kubernetes Ingress nginx配置 (16)