无监控不运维,下面我们就来安装prometheus operator用来监控k8s的内部情况。
我用的是Kubernetes v1.23.8安装prometheus operator 0.11.0版本
版本如下(注意需要注意prometheus operator对应的版本兼容)
Kubernetes v1.23.8
prometheus operator 0.11.0
官方地址
https://github.com/prometheus-operator/kube-prometheus
下载好prometheus operator后解压
cd manifests/

修改持久化

首先你要创建一个存储类,这里因为有了一个监控了,就不贴出yaml了。
我的storageClassName名字叫nfs-prometheus
增加一个storage

[root@az-k8s-nginx-001 ]$vim prometheus-prometheus.yaml
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
metadata:
  labels:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: k8s
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 2.36.1
  name: k8s
  namespace: monitoring
spec:
  alerting:
    alertmanagers:
    - apiVersion: v2
      name: alertmanager-main
      namespace: monitoring
      port: web
  enableFeatures: []
  externalLabels: {}
  image: quay.io/prometheus/prometheus:v2.36.1
  nodeSelector:
    kubernetes.io/os: linux
  podMetadata:
    labels:
      app.kubernetes.io/component: prometheus
      app.kubernetes.io/instance: k8s
      app.kubernetes.io/name: prometheus
      app.kubernetes.io/part-of: kube-prometheus
      app.kubernetes.io/version: 2.36.1
  podMonitorNamespaceSelector: {}
  podMonitorSelector: {}
  probeNamespaceSelector: {}
  probeSelector: {}
  replicas: 2
  resources:
    requests:
      memory: 400Mi
  ruleNamespaceSelector: {}
  ruleSelector: {}
  securityContext:
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName: prometheus-k8s
  serviceMonitorNamespaceSelector: {}
  serviceMonitorSelector: {}
  version: 2.36.1
  storage:
    volumeClaimTemplate:
      spec:
        storageClassName: nfs-prometheus
        resources:
          requests:
            storage: 10Gi

修改grafana访问方式为nodeport,你也可以使用官方的方式。

vim grafana-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: grafana
    app.kubernetes.io/name: grafana
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 8.5.5
  name: grafana
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: http
    port: 3000
    targetPort: http
    nodePort: 31010
  selector:
    app.kubernetes.io/component: grafana
    app.kubernetes.io/name: grafana
    app.kubernetes.io/part-of: kube-prometheus

修改prometheus访问方式为nodeport,你也可以使用官方的方式。

vim prometheus-service.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: k8s
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 2.36.1
  name: prometheus-k8s
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: web
    port: 9090
    targetPort: web
    nodePort: 31011
  - name: reloader-web
    port: 8080
    targetPort: reloader-web
  selector:
    app.kubernetes.io/component: prometheus
    app.kubernetes.io/instance: k8s
    app.kubernetes.io/name: prometheus
    app.kubernetes.io/part-of: kube-prometheus
  sessionAffinity: ClientIP

alertmanager配置nodeport方式方便报警调试

vim alertmanager-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: alert-router
    app.kubernetes.io/instance: main
    app.kubernetes.io/name: alertmanager
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 0.24.0
  name: alertmanager-main
  namespace: monitoring
spec:
  type: NodePort
  ports:
  - name: web
    port: 9093
    targetPort: web
    nodePort: 31012
  - name: reloader-web
    port: 8080
    targetPort: reloader-web
  selector:
    app.kubernetes.io/component: alert-router
    app.kubernetes.io/instance: main
    app.kubernetes.io/name: alertmanager
    app.kubernetes.io/part-of: kube-prometheus
  sessionAffinity: ClientIP

替换镜像地址(由于国内无法访问国外的源需要替换)

cd ./kube-prometheus/manifests/
sed -i 's/quay.io/quay.mirrors.ustc.edu.cn/g' prometheus-prometheus.yaml
sed -i 's/quay.io/quay.mirrors.ustc.edu.cn/g' alertmanager-alertmanager.yaml
sed -i 's/quay.io/quay.mirrors.ustc.edu.cn/g' blackboxExporter-deployment.yaml
sed -i 's/quay.io/quay.mirrors.ustc.edu.cn/g' nodeExporter-daemonset.yaml
sed -i 's/quay.io/quay.mirrors.ustc.edu.cn/g' prometheusOperator-deployment.yaml

这里需要特别注意,使用的是自己私有镜像

sed -i 's/k8s.gcr.io\/prometheus-adapter/registry.cn-hangzhou.aliyuncs.com\/yx-k8s/g' prometheusAdapter-deployment.yaml
sed -i 's/k8s.gcr.io\/kube-state-metrics/registry.cn-hangzhou.aliyuncs.com\/yx-k8s/g' kubeStateMetrics-deployment.yaml
sed -i 's/quay.io\/brancz/registry.cn-hangzhou.aliyuncs.com\/yx-k8s/g' kubeStateMetrics-deployment.yaml

更改客户端监控的9100地址为9200(原因是我本地使用了一个9100的端口避免冲突)

sed -i s/9100/9200/g *

执行安装命令

kubectl apply --server-side -f manifests/setup
kubectl apply -f manifests/

然后检查

until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done
NAMESPACE    NAME                      AGE
monitoring   alertmanager-main         37m
monitoring   blackbox-exporter         37m
monitoring   coredns                   37m
monitoring   grafana                   37m
monitoring   kube-apiserver            37m
monitoring   kube-controller-manager   37m
monitoring   kube-scheduler            37m
monitoring   kube-state-metrics        37m
monitoring   kubelet                   37m
monitoring   node-exporter             37m
monitoring   prometheus-adapter        37m
monitoring   prometheus-k8s            37m
monitoring   prometheus-operator       37m

检查pod状态如图
2022-08-04T08:07:41.png

访问prometheus地址ip加端口我的是ip:31011效果如下
2022-08-04T08:10:34.png
访问Grafana地址ip加端口我的是ip:31010效果如下
2022-08-04T08:12:56.png

由于无法下载国外镜像我的方法是使用docker save和docker load -i命令使用文件

先到能下载的机器上pull下来

docker pull willdockerhub/prometheus-adapter:v0.9.1
docker pull bitnami/kube-rbac-proxy:0.12.0

然后保持为文件

docker save willdockerhub/prometheus-adapter:v0.9.1 > prometheus-adapter.tar
docker save docker.io/bitnami/kube-state-metrics:2.5.0 > kube-state-metrics.tar

加载到机器中去

docker load -i kube-state-metrics.tar
docker load -i prometheus-adapter.tar 

docker tag  bitnami/kube-state-metrics:2.5.0 registry.cn-hangzhou.aliyuncs.com/yx-k8s/kube-state-metrics:v2.5.0 
docker tag willdockerhub/prometheus-adapter:v0.9.1 registry.cn-hangzhou.aliyuncs.com/yx-k8s/prometheus-adapter:v0.9.1

镜像推送

docker push registry.cn-hangzhou.aliyuncs.com/yx-k8s/kube-state-metrics:v2.5.0
docker push registry.cn-hangzhou.aliyuncs.com/yx-k8s/prometheus-adapter:v0.9.1

然后替换成自己的地址国内就可以使用了。
到此完成。

Last modification:August 5, 2022
如果觉得我的文章对你有用,请随意赞赏