基于Docker搭建多主多从K8s高可用集群 - 在线方式 - 永久证书主机名IP地址角色操作系统硬件配置ansible10.62.158.200同步工具节点CentOS 72 Core/4G Memorymaster0110.62.158.201管理节点01CentOS 72 Core/4G Memorymaster0210.62.158.202管理节点02CentOS 72 Core/4G Memorymaster0310.62.158.203管理节点03CentOS 72 Core/4G Memorynode0110.62.158.204工作节点01CentOS 71 Core/2G Memorynode0210.62.158.205工作节点02CentOS 71 Core/2G Memoryk8s-ha0110.62.158.206主代理节点CentOS 71 Core/2G Memoryk8s-ha0210.62.158.207备用代理节点CentOS 71 Core/2G Memory查看当前集群状态[rootmaster01 ~]# kubectl get nodeNAME STATUS ROLES AGE VERSION master01 Ready control-plane,master 22d v1.23.0 master02 Ready control-plane,master 22d v1.23.0 master03 Ready control-plane,master 22d v1.23.0 node01 Ready none 22d v1.23.0 node02 Ready none 22d v1.23.0查看证书过期时间[rootmaster01 ~]# kubeadm certs check-expiration[check-expiration]Reading configurationfromthe cluster...[check-expiration]FYI: You can look at this config file withkubectl -n kube-system get cm kubeadm-config -o yamlCERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Apr 17,2025 04:39 UTC 342d no apiserver Apr 17,2025 04:39 UTC 342d ca no apiserver-etcd-client Apr 17,2025 04:39 UTC 342d etcd-ca no apiserver-kubelet-client Apr 17,2025 04:39 UTC 342d ca no controller-manager.conf Apr 17,2025 04:39 UTC 342d no etcd-healthcheck-client Apr 17,2025 04:39 UTC 342d etcd-ca no etcd-peer Apr 17,2025 04:39 UTC 342d etcd-ca no etcd-server Apr 17,2025 04:39 UTC 342d etcd-ca no front-proxy-client Apr 17,2025 04:39 UTC 342d front-proxy-ca no scheduler.conf Apr 17,2025 04:39 UTC 342d no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Apr 15,2034 04:39 UTC 9y no etcd-ca Apr 15,2034 04:39 UTC 9y no front-proxy-ca Apr 15,2034 04:39 UTC 9y noGoLang环境安装 - go1.22.0.linux-amd64.tar.gz# 下载环境文件[rootmaster01 ~]# wget https://studygolang.com/dl/golang/go1.17.6.linux-amd64.tar.gz[rootmaster01 ~]# lsanaconda-ks.cfg calico.yaml docker docker-20.10.tar.gz go1.17.6.linux-amd64.tar.gz kubeadm-config.yml nginx.yml sysconfigure.sh# 文件解压[rootmaster01 ~]# tar -xvf go1.17.6.linux-amd64.tar.gz -C /usr/local/# 配置环境变量[rootmaster01 ~]# echo export PATH$PATH:/usr/local/go/bin /etc/profile[rootmaster01 ~]# source /etc/profile# 验证go环境[rootmaster01 ~]# go versiongo version go1.17.6 linux/amd64下载kubernetes源码 - kubernetes-1.23.0.tar.gz# 查看下当前已安装的k8s版本[rootmaster01 ~]# kubectl versionClient Version: version.Info{Major:1,Minor:23,GitVersion:v1.23.0,GitCommit:ab69524f795c42094a6630298ff53f3c3ebab7f4,GitTreeState:clean,BuildDate:2021-12-07T18:16:20Z,GoVersion:go1.17.3,Compiler:gc,Platform:linux/amd64}Server Version: version.Info{Major:1,Minor:23,GitVersion:v1.23.0,GitCommit:ab69524f795c42094a6630298ff53f3c3ebab7f4,GitTreeState:clean,BuildDate:2021-12-07T18:09:57Z,GoVersion:go1.17.3,Compiler:gc,Platform:linux/amd64}# 下载对应的源码[rootmaster01 ~]# wget https://github.com/kubernetes/kubernetes/archive/v1.23.0.tar.gz[rootmaster01 ~]# lsanaconda-ks.cfg calico.yaml docker docker-20.10.tar.gz go1.17.6.linux-amd64.tar.gz kubeadm-config.yml kubernetes-1.23.0.tar.gz nginx.yml sysconfigure.sh# 源码解压[rootmaster01 ~]# tar -zxvf kubernetes-1.23.0.tar.gz[rootmaster01 ~]# lsanaconda-ks.cfg calico.yaml docker docker-20.10.tar.gz go1.17.6.linux-amd64.tar.gz kubeadm-config.yml kubernetes-1.23.0 kubernetes-1.23.0.tar.gz nginx.yml sysconfigure.sh修改证书有效期[rootmaster01 ~]# cd kubernetes-1.23.0[rootmaster01 kubernetes-1.23.0]# vim ./cmd/kubeadm/app/constants/constants.goCertificateValidity time.Hour*24*365*100[rootmaster kubernetes-1.23.0]# vim staging/src/k8s.io/client-go/util/cert/cert.goNotAfter: now.Add(duration365d*100).UTC(),编译源代码文件[rootmaster01 kubernetes-1.23.0]# make WHATcmd/kubeadm GOFLAGS-v注意若出现 ./hack/run-in-gopath.sh:行34: _output/bin/prerelease-lifecycle-gen: 权限不够 问题时需要添加权限后再编译源码文件[rootmaster01 kubernetes-1.23.0]# yum install rsync jq -y[rootmaster01 bin]# chmod x _output/bin/prerelease-lifecycle-gen[rootmaster01 bin]# chmod x _output/bin/deepcopy-gen查看编译后kubeadm二进制文件 - kubeadm[rootmaster01 kubernetes-1.23.0]# ls -l _output/bin/总用量 79012-rwxr-xr-x 1 root root 6275072 5月 9 18:19 conversion-gen-rwxr-xr-x 1 root root 5996544 5月 9 18:19 deepcopy-gen-rwxr-xr-x 1 root root 6000640 5月 9 18:19 defaulter-gen-rwxr-xr-x 1 root root 3376695 5月 9 18:19 go2make-rwxr-xr-x 1 root root 45170688 5月 9 18:43 kubeadm-rwxr-xr-x 1 root root 8114176 5月 9 18:19 openapi-gen-rwxr-xr-x 1 root root 5971968 5月 9 18:19 prerelease-lifecycle-gen备份原有的kubeadm文件[rootmaster01 kubernetes-1.23.0]# cp /usr/bin/kubeadm /usr/bin/kubeadm_bak20240509替换新编译的kubeadm文件覆盖旧的kubeadm文件# 集群初始化时仅使用kubeadm二进制文件即可需要赋予可执行权限[rootmaster01 kubernetes-1.23.0]# cp /root/kubernetes-1.23.0/_output/bin/kubeadm /usr/bin/备份pki证书文件[rootmaster01 kubernetes-1.23.0]# cd /etc/kubernetes[rootmaster01 kubernetes]# cp -R pki/ pki_bak20240509 -- 未初始化不需要这个证书更新# CA根证书不生效需要集群初始化时才生效[rootmaster01 kubernetes]# kubeadm certs renew all[renew]Reading configurationfromthe cluster...[renew]FYI: You can look at this config file withkubectl -n kube-system get cm kubeadm-config -o yamlcertificate embedded in the kubeconfig fileforthe admin to use andforkubeadm itself renewed certificateforserving the Kubernetes API renewed certificate the apiserver uses to access etcd renewed certificateforthe API server to connect to kubelet renewed certificate embedded in the kubeconfig fileforthe controller manager to use renewed certificateforliveness probes to healthcheck etcd renewed certificateforetcd nodes to communicate with each other renewed certificateforserving etcd renewed certificateforthe front proxy client renewed certificate embedded in the kubeconfig fileforthe scheduler manager to use renewed Done renewing certificates.You must restart the kube-apiserver,kube-controller-manager,kube-scheduler and etcd,so that they can use the new certificates.查看证书是否为100年[rootmaster01 kubernetes]# kubeadm certs check-expiration[check-expiration]Reading configurationfromthe cluster...[check-expiration]FYI: You can look at this config file withkubectl -n kube-system get cm kubeadm-config -o yamlCERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Apr 15,2124 11:45 UTC 99y no apiserver Apr 15,2124 11:45 UTC 99y ca no apiserver-etcd-client Apr 15,2124 11:45 UTC 99y etcd-ca no apiserver-kubelet-client Apr 15,2124 11:45 UTC 99y ca no controller-manager.conf Apr 15,2124 11:45 UTC 99y no etcd-healthcheck-client Apr 15,2124 11:45 UTC 99y etcd-ca no etcd-peer Apr 15,2124 11:45 UTC 99y etcd-ca no etcd-server Apr 15,2124 11:45 UTC 99y etcd-ca no front-proxy-client Apr 15,2124 11:45 UTC 99y front-proxy-ca no scheduler.conf Apr 15,2124 11:45 UTC 99y no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Apr 15,2034 04:39 UTC 9y no etcd-ca Apr 15,2034 04:39 UTC 9y no front-proxy-ca Apr 15,2034 04:39 UTC 9y no上传01管理节点中编译后的kubeadm二进制文件到02管理节点和03管理节点02管理节点与03管理节点证书有效期调整操作步骤一致以下为02管理节点操作过程上传kubeadm二进制文件[rootmaster02 ~]# lsanaconda-ks.cfg docker docker-20.10.tar.gz kubeadm sysconfigure.sh添加执行权限[rootmaster02 ~]# chmod x kubeadm备份原有的kubeadm文件[rootmaster02 ~]# cp /usr/bin/kubeadm /usr/bin/kubeadm_bak20240509替换新编译的kubeadm文件覆盖旧的kubeadm文件[rootmaster kubernetes-1.23.0]# cp /root/kubeadm /usr/bin/备份pki证书文件[rootmaster02 ~]# cd /etc/kubernetes[rootmaster02 kubernetes]# cp -R pki/ pki_bak20240509证书更新[rootmaster02 kubernetes]# kubeadm certs renew all[renew]Reading configurationfromthe cluster...[renew]FYI: You can look at this config file withkubectl -n kube-system get cm kubeadm-config -o yamlcertificate embedded in the kubeconfig fileforthe admin to use andforkubeadm itself renewed certificateforserving the Kubernetes API renewed certificate the apiserver uses to access etcd renewed certificateforthe API server to connect to kubelet renewed certificate embedded in the kubeconfig fileforthe controller manager to use renewed certificateforliveness probes to healthcheck etcd renewed certificateforetcd nodes to communicate with each other renewed certificateforserving etcd renewed certificateforthe front proxy client renewed certificate embedded in the kubeconfig fileforthe scheduler manager to use renewed Done renewing certificates.You must restart the kube-apiserver,kube-controller-manager,kube-scheduler and etcd,so that they can use the new certificates.查看证书是否为100年[rootmaster02 kubernetes]# kubeadm certs check-expiration[check-expiration]Reading configurationfromthe cluster...[check-expiration]FYI: You can look at this config file withkubectl -n kube-system get cm kubeadm-config -o yamlCERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Apr 15,2124 11:56 UTC 99y no apiserver Apr 15,2124 11:56 UTC 99y ca no apiserver-etcd-client Apr 15,2124 11:56 UTC 99y etcd-ca no apiserver-kubelet-client Apr 15,2124 11:56 UTC 99y ca no controller-manager.conf Apr 15,2124 11:56 UTC 99y no etcd-healthcheck-client Apr 15,2124 11:56 UTC 99y etcd-ca no etcd-peer Apr 15,2124 11:56 UTC 99y etcd-ca no etcd-server Apr 15,2124 11:56 UTC 99y etcd-ca no front-proxy-client Apr 15,2124 11:56 UTC 99y front-proxy-ca no scheduler.conf Apr 15,2124 11:56 UTC 99y no CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED ca Apr 15,2034 04:39 UTC 9y no etcd-ca Apr 15,2034 04:39 UTC 9y no front-proxy-ca Apr 15,2034 04:39 UTC 9y no