一: 简介1.0 什么是探针livenessProbe存活探针 是 Kubernetes 中最核心的容器健康检查机制 用于判断容器内的应用是否 “存活”正常运行。 如果探测失败Kubernetes 会按照 Pod 的重启策略restartPolicy 自动重启容器实现故障自愈。这里的探测失败不是指探测过程失败。而指的是探测的结果不符合预期实际上就是检测容器健康的机制判断容器是否正常运行如果探测失败(返回结果不符合预期)就需要重启容器从而实现故障自愈2.0 为啥需要探针没有探针进程活着但服务坏了我们的k8s发现不了问题容器不重启有探针(前提是这个探针检测的就是这个服务)进程活着但服务坏了也能发现并重启自动修复不用等人工介入3.0 探针的分类二: 命令模式探针1.0 准备并创建探针准备探针[rootk8s-master ~]# mkdir 10livenessProbe ; cd 10livenessProbe/[rootk8s-master 10livenessProbe]# vim test-liveness-exec.yaml--- apiVersion: v1 kind: Pod metadata: labels: test: liveness#标签名字自定义name: test-liveness-exec#名字自定义# 表示我要创建一个pod,他的名字是test-liveness-exec。给他贴身一个标签名字是test,值为livenessspec: containers: - name: liveness#名字自定义image: nginx args:#模拟程序的健康和故障- /bin/sh#sh -c 非交互式执行命令--c-touch/tmp/healthy;sleep30;rm-rf/tmp/healthy;sleep50# 由于我们在下面设置每5秒检测一下状态k8s默认3次之后就直接重启容器了。 sleep 50 不是等 50 秒再重启而是让容器多活 50 秒给探针创造检测和杀死的时机。# spec部分为容器的具体特性首先是容器的基本信息再是args表示“先健康后故障的剧本livenessProbe:#探针健康检查exec:#类型command:#命令-cat- /tmp/healthy initialDelaySeconds:5#健康检查在容器启动5s后开始执行periodSeconds:5#每5s执行一次restartPolicy: Always#探针失败后自动重启容器线上建议Always# livenessProbe: 在容器内部每五秒执行一次cat /tmp/healthy。 如果存在测探测成功如果不存在表示探测失败创建pod[rootk8s-master 10livenessProbe]# kubectl apply -f test-liveness-exec.yaml2.0 查看pod状态查看pod状态[rootk8s-master 10livenessProbe]# kubectl get podNAME READY STATUS RESTARTS AGE my-mysql1/1 Running315d test-liveness-exec1/1 Running013s[rootk8s-master 10livenessProbe]#详细查看pod状态[rootk8s-master 10livenessProbe]# kubectl describe pod test-liveness-execEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 30s default-scheduler Successfully assigned default/test-liveness-exec to k8s-node2 Normal Pulling 29s kubelet Pulling imagenginxNormal Pulled 19s kubelet Successfully pulled imagenginxin9.743877102s Normal Created 19s kubelet Created container liveness Normal Started 18s kubelet Started container liveness[rootk8s-master 10livenessProbe]#详细查看pod状态Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 68s default-scheduler Successfully assigned default/test-liveness-exec to k8s-node2 Normal Pulling 67s kubelet Pulling imagenginxNormal Pulled 57s kubelet Successfully pulled imagenginxin9.743877102s Normal Created 57s kubelet Created container liveness Normal Started 56s kubelet Started container liveness Warning Unhealthy 14s(x3 over 24s)kubelet Liveness probe failed: cat: /tmp/healthy: No suchfileor directory Normal Killing 14s kubelet Container liveness failed liveness probe, will be restarted[rootk8s-master 10livenessProbe]#查看pod状态[rootk8s-master 10livenessProbe]# kubectl get pod test-liveness-execNAME READY STATUS RESTARTS AGE test-liveness-exec1/1 Running12m2s[rootk8s-master 10livenessProbe]#[rootk8s-master 10livenessProbe]# kubectl get pod test-liveness-execNAME READY STATUS RESTARTS AGE test-liveness-exec1/1 Running34m25s[rootk8s-master 10livenessProbe]#[rootk8s-master 10livenessProbe]# kubectl get pod test-liveness-execNAME READY STATUS RESTARTS AGE test-liveness-exec1/1 Running45m30s[rootk8s-master 10livenessProbe]#3.0 总结我们可以发现容器一直处于重启的状态中。 这个原因是因为我们在容器中删除了/tmp/health文件但是我们探测的内容就是这个文件可以发现探测失败了因此一直重启容器。直到我们手动干预在容器中新建这个文件。三: http get方式探针1.0 准备并创建探针定义pod[rootk8s-master 10livenessProbe]# vim liveness-httpget.yaml--- apiVersion: v1 kind: Pod metadata: name: liveness-httpget-pod namespace: default spec: containers: - name: liveness-exec-container image: nginx imagePullPolicy: IfNotPresent ports: - name: http containerPort:80livenessProbe:#探针健康检查httpGet: port: http path: /index.html initialDelaySeconds:1#容器启动后等待 1 秒就开始第一次存活检测periodSeconds:3#第一次检测后每隔 3 秒执行一次存活检测# Kubernetes 会每隔 3 秒向容器内的 http://容器IP:端口/index.html 发起一个 HTTP GET 请求创建pod[rootk8s-master 10livenessProbe]# kubectl apply -f liveness-httpget.yaml2.0 查看查看pod[rootk8s-master 10livenessProbe]# kubectl get podsNAME READY STATUS RESTARTS AGE liveness-httpget-pod1/1 Running012s my-mysql1/1 Running315d[rootk8s-master 10livenessProbe]#[rootk8s-master 10livenessProbe]# kubectl describe pod liveness-httpget-podvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 30s default-scheduler Successfully assigned default/liveness-httpget-pod to k8s-node1 Normal Pulled 28s kubelet Container imagenginxalready present on machine Normal Created 28s kubelet Created container liveness-exec-container Normal Started 27s kubelet Started container liveness-exec-container[rootk8s-master 10livenessProbe]#登录容器测试故障登陆容器测试将容器内的index.html删除掉[rootkub-k8s-master 10livenessProbe]# kubectl exec -it liveness-httpget-pod /bin/bashrootliveness-httpget-pod:/# mv /usr/share/nginx/html/index.html index.html容器退出提示 rootliveness-httpget-pod:/# command terminated with exit code 137可以看到当把index.html移走后这个容器立马就退出了。再次查看[rootk8s-master 10livenessProbe]# kubectl describe pod liveness-httpget-podEvents: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 3m19s default-scheduler Successfully assigned default/liveness-httpget-pod to k8s-node1 Warning Unhealthy 65s(x3 over 71s)kubelet Liveness probe failed: HTTP probe failed with statuscode:404Normal Killing 65s kubelet Container liveness-exec-container failed liveness probe, will be restarted Normal Pulled 64s(x2 over 3m17s)kubelet Container imagenginxalready present on machine Normal Created 64s(x2 over 3m17s)kubelet Created container liveness-exec-container Normal Started 64s(x2 over 3m16s)kubelet Started container liveness-exec-container[rootk8s-master 10livenessProbe]#[rootk8s-master 10livenessProbe]# kubectl get pod liveness-httpget-pod -o wide登录容器测试故障重新登陆容器发现index.html又出现了 证明容器是被重拉了。探针测试成功[rootk8s-master 10livenessProbe]# kubectl exec -it liveness-httpget-pod /bin/bashrootliveness-httpget-pod:/# cat /usr/share/nginx/html/index.html重新生成的index.html是nginx默认的测试页设置。 所以生产环境下健康探针应监视健康测试页或其他页面四: 三大探针区别三类探针的核心区别LivenessProbe管生死→ 死了就重启(各个APP必备)ReadinessProbe管能不能干活→ 不能干活就不让请求进来大服务如mysqlStartupProbe管启动完没→ 没启动完就不打扰不执行其他探针。超大服务如机器学习服务