前面说了prometheus operator持久化的问题,但是还有一个问题很多人都忽略了,那就是prometheus operator数据保留天数,根据官方文档的说明,默认prometheus operator数据存储的时间为1d,这个时候无论你prometheus operator如何进行持久化,都没有作用,因为数据只保留了1天,那么你是无法看到更多天数的数据
Prometheus
关于prometheus operator更多监控可以参考下面的文章
Prometheus Operator 监控k8s组件
新闻联播老司机
Prometheus Operator
新闻联播老司机
官方文档可以配置的说明
图片来自官方文档
如果你的prometheus operator想使用NFS持久化可以看下面的文章,和本次修改prometheus operator默认持久化数据不冲突
Prometheus Operator 持久化存储
新闻联播老司机
实际上修改prometheus operator时间是通过retention参数进行修改,上面也提示了在prometheus.spec下填写
#接下来进行填写,首先我们先下载operator,并将目录进行分类 wget -P /root/ http://down.i4t.com/abcdocker-prometheus-operator.yaml.zip cd /root/ unzip abcdocker-prometheus-operator.yaml.zip mkdir kube-prom cp -a kube-prometheus-master/manifests/* kube-prom/ cd kube-prom/ mkdir -p node-exporter alertmanager grafana kube-state-metrics prometheus serviceMonitor adapter operator mv *-serviceMonitor* serviceMonitor/ mv setup operator/ mv grafana-* grafana/ mv kube-state-metrics-* kube-state-metrics/ mv alertmanager-* alertmanager/ mv node-exporter-* node-exporter/ mv prometheus-adapter* adapter/ mv prometheus-* prometheus/ mv 0prometheus-operator-* operator/ mv 00namespace-namespace.yaml operator/ #接下来需要修改prometheus operator的deployment文件 cd /root/kube-prom/prometheus/ vim prometheus-prometheus.yaml 在spec.下面添加 retention: 7d
如果已经安装了可以直接修改prometheus-prometheus.yaml 然后通过kubectl apply -f 刷新即可
修改完毕后检查pod运行状态是否正常
[root@YZSJHL82-101 ~]# kubectl get all -n monitoring NAME READY STATUS RESTARTS AGE pod/alertmanager-main-0 2/2 Running 0 44h pod/alertmanager-main-1 2/2 Running 0 44h pod/alertmanager-main-2 2/2 Running 0 44h pod/grafana-558647b59-tlp8s 1/1 Running 0 44h pod/kube-state-metrics-5bfc7db74d-8d4jk 4/4 Running 0 44h pod/node-exporter-jwl9v 2/2 Running 0 44h pod/node-exporter-nxn2w 2/2 Running 0 44h pod/node-exporter-s6lpr 2/2 Running 0 44h pod/prometheus-adapter-57c497c557-svfs8 1/1 Running 0 44h pod/prometheus-k8s-0 3/3 Running 1 44h pod/prometheus-k8s-1 3/3 Running 1 44h pod/prometheus-operator-69bd579bf9-tzr5z 1/1 Running 0 44h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/alertmanager-main ClusterIP 10.254.149.37 9093/TCP 44h service/alertmanager-operated ClusterIP None 9093/TCP,6783/TCP 44h service/grafana NodePort 10.254.134.244 3000:20614/TCP 44h service/kube-state-metrics ClusterIP None 8443/TCP,9443/TCP 44h service/node-exporter ClusterIP None 9100/TCP 44h service/prometheus-adapter ClusterIP 10.254.7.111 443/TCP 44h service/prometheus-k8s NodePort 10.254.130.51 9090:13535/TCP 44h service/prometheus-operated ClusterIP None 9090/TCP 44h service/prometheus-operator ClusterIP None 8080/TCP 44h NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/node-exporter 3 3 3 3 3 beta.kubernetes.io/os=linux 44h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/grafana 1/1 1 1 44h deployment.apps/kube-state-metrics 1/1 1 1 44h deployment.apps/prometheus-adapter 1/1 1 1 44h deployment.apps/prometheus-operator 1/1 1 1 44h NAME DESIRED CURRENT READY AGE replicaset.apps/grafana-558647b59 1 1 1 44h replicaset.apps/kube-state-metrics-5bfc7db74d 1 1 1 44h replicaset.apps/prometheus-adapter-57c497c557 1 1 1 44h replicaset.apps/prometheus-operator-69bd579bf9 1 1 1 44h NAME READY AGE statefulset.apps/alertmanager-main 3/3 44h statefulset.apps/prometheus-k8s 2/2 44h
接下来可以访问grafana或者prometheus ui进行检查 (我这里修改完毕后等待2天,检查数据是否正常)
修改前
修改后
相关文章:
- Kubernetes 1.14 二进制集群安装
- Prometheus Operator 监控k8s组件
- Kubenetes 1.13.5 集群二进制安装
- Kuerbernetes 1.11 集群二进制安装