1. Kubernetes V1.33二进制部署实战指南Rocky Linux环境在容器编排领域Kubernetes已经成为事实上的标准。虽然现在有很多自动化部署工具如kubeadm但通过二进制方式手动部署仍然是理解Kubernetes架构最彻底的方式。本文将基于Rocky Linux 8系统详细演示如何从零开始部署Kubernetes V1.33集群。提示二进制部署方式虽然步骤繁琐但能让你对Kubernetes各组件的交互关系有更深刻的理解特别适合想要深入掌握Kubernetes原理的运维人员和开发者。1.1 环境准备与规划在开始之前我们需要准备至少两台Rocky Linux 8服务器物理机或虚拟机均可1台作为Master节点建议4核CPU/8GB内存/50GB磁盘1台或更多作为Worker节点建议2核CPU/4GB内存/30GB磁盘网络规划建议所有节点需要能互相通信建议关闭防火墙或配置放行规则为集群预留独立的网段如10.244.0.0/16为Service预留网段如10.96.0.0/12首先在所有节点执行基础环境配置# 关闭SELinux setenforce 0 sed -i s/^SELINUXenforcing$/SELINUXpermissive/ /etc/selinux/config # 关闭swap swapoff -a sed -i / swap / s/^\(.*\)$/#\1/g /etc/fstab # 设置主机名分别在对应节点执行 hostnamectl set-hostname k8s-master # Master节点 hostnamectl set-hostname k8s-node1 # Worker节点 # 添加hosts解析 cat /etc/hosts EOF 192.168.1.100 k8s-master 192.168.1.101 k8s-node1 EOF # 加载内核模块 cat /etc/modules-load.d/k8s.conf EOF br_netfilter ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack EOF modprobe br_netfilter modprobe ip_vs modprobe ip_vs_rr modprobe ip_vs_wrr modprobe ip_vs_sh modprobe nf_conntrack # 设置内核参数 cat /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 EOF sysctl --system1.2 二进制文件下载与分发从官方GitHub仓库下载所需版本的二进制文件以v1.33.0为例# 在Master节点执行 wget https://dl.k8s.io/v1.33.0/kubernetes-server-linux-amd64.tar.gz tar -zxvf kubernetes-server-linux-amd64.tar.gz cd kubernetes/server/bin cp kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/ # 在所有节点执行包括Worker wget https://dl.k8s.io/v1.33.0/kubernetes-node-linux-amd64.tar.gz tar -zxvf kubernetes-node-linux-amd64.tar.gz cd kubernetes/node/bin cp kubelet kube-proxy /usr/local/bin/1.3 证书生成与配置Kubernetes各个组件之间采用TLS加密通信我们需要先创建CA证书和各个组件的证书# 安装cfssl工具 wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O /usr/local/bin/cfssl wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O /usr/local/bin/cfssljson chmod x /usr/local/bin/cfssl* # 创建CA配置文件 cat ca-config.json EOF { signing: { default: { expiry: 87600h }, profiles: { kubernetes: { usages: [signing, key encipherment, server auth, client auth], expiry: 87600h } } } } EOF # 创建CA证书签名请求 cat ca-csr.json EOF { CN: Kubernetes, key: { algo: rsa, size: 2048 }, names: [ { C: CN, L: Beijing, O: Kubernetes, OU: CA, ST: Beijing } ] } EOF # 生成CA证书和私钥 cfssl gencert -initca ca-csr.json | cfssljson -bare ca接下来为各个组件生成证书以apiserver为例# 创建apiserver证书请求 cat kube-apiserver-csr.json EOF { CN: kube-apiserver, hosts: [ 127.0.0.1, 192.168.1.100, 10.96.0.1, kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster, kubernetes.default.svc.cluster.local ], key: { algo: rsa, size: 2048 }, names: [ { C: CN, L: Beijing, O: Kubernetes, OU: Kubernetes, ST: Beijing } ] } EOF # 生成apiserver证书 cfssl gencert -caca.pem -ca-keyca-key.pem -configca-config.json -profilekubernetes kube-apiserver-csr.json | cfssljson -bare kube-apiserver注意需要为每个组件etcd、kubelet等分别生成证书过程类似只是CN和hosts字段需要相应调整。1.4 部署Master节点组件1.4.1 部署etcd集群# 下载etcd二进制文件 wget https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz tar -zxvf etcd-v3.5.0-linux-amd64.tar.gz cp etcd-v3.5.0-linux-amd64/etcd* /usr/local/bin/ # 创建etcd配置文件 cat /etc/etcd/etcd.conf EOF ETCD_NAMEetcd1 ETCD_DATA_DIR/var/lib/etcd ETCD_LISTEN_PEER_URLShttps://192.168.1.100:2380 ETCD_LISTEN_CLIENT_URLShttps://192.168.1.100:2379,https://127.0.0.1:2379 ETCD_INITIAL_ADVERTISE_PEER_URLShttps://192.168.1.100:2380 ETCD_ADVERTISE_CLIENT_URLShttps://192.168.1.100:2379 ETCD_INITIAL_CLUSTERetcd1https://192.168.1.100:2380 ETCD_INITIAL_CLUSTER_TOKENetcd-cluster ETCD_INITIAL_CLUSTER_STATEnew ETCD_CERT_FILE/etc/kubernetes/pki/etcd.pem ETCD_KEY_FILE/etc/kubernetes/pki/etcd-key.pem ETCD_PEER_CERT_FILE/etc/kubernetes/pki/etcd.pem ETCD_PEER_KEY_FILE/etc/kubernetes/pki/etcd-key.pem ETCD_TRUSTED_CA_FILE/etc/kubernetes/pki/ca.pem ETCD_PEER_TRUSTED_CA_FILE/etc/kubernetes/pki/ca.pem EOF # 创建systemd服务 cat /usr/lib/systemd/system/etcd.service EOF [Unit] DescriptionEtcd Server Afternetwork.target [Service] Typenotify EnvironmentFile-/etc/etcd/etcd.conf ExecStart/usr/local/bin/etcd Restarton-failure LimitNOFILE65536 [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable etcd systemctl start etcd1.4.2 部署kube-apiserver# 创建配置文件 cat /etc/kubernetes/kube-apiserver.conf EOF KUBE_APISERVER_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/var/log/kubernetes \\ --etcd-servershttps://192.168.1.100:2379 \\ --bind-address0.0.0.0 \\ --secure-port6443 \\ --advertise-address192.168.1.100 \\ --allow-privilegedtrue \\ --service-cluster-ip-range10.96.0.0/12 \\ --enable-admission-pluginsNodeRestriction \\ --authorization-modeRBAC,Node \\ --enable-bootstrap-token-authtrue \\ --token-auth-file/etc/kubernetes/token.csv \\ --service-node-port-range30000-32767 \\ --tls-cert-file/etc/kubernetes/pki/kube-apiserver.pem \\ --tls-private-key-file/etc/kubernetes/pki/kube-apiserver-key.pem \\ --client-ca-file/etc/kubernetes/pki/ca.pem \\ --kubelet-client-certificate/etc/kubernetes/pki/apiserver-kubelet-client.pem \\ --kubelet-client-key/etc/kubernetes/pki/apiserver-kubelet-client-key.pem \\ --service-account-key-file/etc/kubernetes/pki/sa.pub \\ --service-account-signing-key-file/etc/kubernetes/pki/sa.key \\ --service-account-issuerhttps://kubernetes.default.svc.cluster.local \\ --etcd-cafile/etc/kubernetes/pki/ca.pem \\ --etcd-certfile/etc/kubernetes/pki/apiserver-etcd-client.pem \\ --etcd-keyfile/etc/kubernetes/pki/apiserver-etcd-client-key.pem \\ --requestheader-client-ca-file/etc/kubernetes/pki/front-proxy-ca.pem \\ --proxy-client-cert-file/etc/kubernetes/pki/front-proxy-client.pem \\ --proxy-client-key-file/etc/kubernetes/pki/front-proxy-client-key.pem \\ --requestheader-allowed-namesfront-proxy-client \\ --requestheader-extra-headers-prefixX-Remote-Extra- \\ --requestheader-group-headers-prefixX-Remote-Group- \\ --requestheader-username-headers-prefixX-Remote-User- \\ --enable-aggregator-routingtrue EOF # 创建systemd服务 cat /usr/lib/systemd/system/kube-apiserver.service EOF [Unit] DescriptionKubernetes API Server Documentationhttps://github.com/kubernetes/kubernetes Afteretcd.service Wantsetcd.service [Service] EnvironmentFile-/etc/kubernetes/kube-apiserver.conf ExecStart/usr/local/bin/kube-apiserver \$KUBE_APISERVER_OPTS Restarton-failure LimitNOFILE65536 [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable kube-apiserver systemctl start kube-apiserver1.4.3 部署kube-controller-manager# 创建配置文件 cat /etc/kubernetes/kube-controller-manager.conf EOF KUBE_CONTROLLER_MANAGER_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/var/log/kubernetes \\ --leader-electtrue \\ --kubeconfig/etc/kubernetes/kube-controller-manager.kubeconfig \\ --bind-address127.0.0.1 \\ --cluster-cidr10.244.0.0/16 \\ --service-cluster-ip-range10.96.0.0/12 \\ --cluster-signing-cert-file/etc/kubernetes/pki/ca.pem \\ --cluster-signing-key-file/etc/kubernetes/pki/ca-key.pem \\ --root-ca-file/etc/kubernetes/pki/ca.pem \\ --service-account-private-key-file/etc/kubernetes/pki/sa.key \\ --use-service-account-credentialstrue \\ --controllers*,bootstrapsigner,tokencleaner EOF # 创建systemd服务 cat /usr/lib/systemd/system/kube-controller-manager.service EOF [Unit] DescriptionKubernetes Controller Manager Documentationhttps://github.com/kubernetes/kubernetes Afterkube-apiserver.service Requireskube-apiserver.service [Service] EnvironmentFile-/etc/kubernetes/kube-controller-manager.conf ExecStart/usr/local/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS Restarton-failure LimitNOFILE65536 [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable kube-controller-manager systemctl start kube-controller-manager1.4.4 部署kube-scheduler# 创建配置文件 cat /etc/kubernetes/kube-scheduler.conf EOF KUBE_SCHEDULER_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/var/log/kubernetes \\ --leader-electtrue \\ --kubeconfig/etc/kubernetes/kube-scheduler.kubeconfig \\ --bind-address127.0.0.1 EOF # 创建systemd服务 cat /usr/lib/systemd/system/kube-scheduler.service EOF [Unit] DescriptionKubernetes Scheduler Documentationhttps://github.com/kubernetes/kubernetes Afterkube-apiserver.service Requireskube-apiserver.service [Service] EnvironmentFile-/etc/kubernetes/kube-scheduler.conf ExecStart/usr/local/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS Restarton-failure LimitNOFILE65536 [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable kube-scheduler systemctl start kube-scheduler1.5 部署Worker节点组件1.5.1 部署kubelet# 创建kubelet配置文件 cat /etc/kubernetes/kubelet.conf EOF KUBELET_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/var/log/kubernetes \\ --hostname-overridek8s-node1 \\ --network-plugincni \\ --kubeconfig/etc/kubernetes/kubelet.kubeconfig \\ --bootstrap-kubeconfig/etc/kubernetes/bootstrap-kubelet.kubeconfig \\ --config/etc/kubernetes/kubelet-config.yml \\ --cert-dir/etc/kubernetes/pki \\ --pod-infra-container-imageregistry.aliyuncs.com/google_containers/pause:3.7 EOF # 创建kubelet-config.yml cat /etc/kubernetes/kubelet-config.yml EOF kind: KubeletConfiguration apiVersion: kubelet.config.k8s.io/v1beta1 address: 0.0.0.0 port: 10250 readOnlyPort: 10255 cgroupDriver: systemd clusterDNS: - 10.96.0.10 clusterDomain: cluster.local failSwapOn: false authentication: anonymous: enabled: false webhook: enabled: true x509: clientCAFile: /etc/kubernetes/pki/ca.pem authorization: mode: Webhook evictionHard: imagefs.available: 15% memory.available: 100Mi nodefs.available: 10% nodefs.inodesFree: 5% maxOpenFiles: 1000000 maxPods: 110 EOF # 创建systemd服务 cat /usr/lib/systemd/system/kubelet.service EOF [Unit] DescriptionKubernetes Kubelet Documentationhttps://github.com/kubernetes/kubernetes Afterdocker.service Requiresdocker.service [Service] EnvironmentFile-/etc/kubernetes/kubelet.conf ExecStart/usr/local/bin/kubelet \$KUBELET_OPTS Restartalways StartLimitInterval0 RestartSec10 [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable kubelet systemctl start kubelet1.5.2 部署kube-proxy# 创建配置文件 cat /etc/kubernetes/kube-proxy.conf EOF KUBE_PROXY_OPTS--logtostderrfalse \\ --v2 \\ --log-dir/var/log/kubernetes \\ --hostname-overridek8s-node1 \\ --cluster-cidr10.244.0.0/16 \\ --kubeconfig/etc/kubernetes/kube-proxy.kubeconfig \\ --proxy-modeipvs \\ --ipvs-schedulerrr EOF # 创建systemd服务 cat /usr/lib/systemd/system/kube-proxy.service EOF [Unit] DescriptionKubernetes Kube Proxy Documentationhttps://github.com/kubernetes/kubernetes [Service] EnvironmentFile-/etc/kubernetes/kube-proxy.conf ExecStart/usr/local/bin/kube-proxy \$KUBE_PROXY_OPTS Restarton-failure LimitNOFILE65536 [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable kube-proxy systemctl start kube-proxy1.6 部署网络插件Calico# 在所有节点安装ipset和ipvsadm yum install ipset ipvsadm -y # 在Master节点部署Calico kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml # 检查部署状态 kubectl get pods -n kube-system1.7 验证集群状态在Master节点执行以下命令验证集群状态kubectl get nodes kubectl get cs kubectl get pods --all-namespaces如果所有组件都正常运行你应该能看到类似以下输出NAME STATUS ROLES AGE VERSION k8s-master Ready master 10m v1.33.0 k8s-node1 Ready none 8m v1.33.01.8 常见问题与解决方案1.8.1 kubelet无法启动问题现象kubelet服务启动失败日志显示Failed to load kubelet config file解决方案检查/etc/kubernetes/kubelet-config.yml文件格式是否正确确保文件权限为644检查YAML文件缩进是否正确1.8.2 Pod网络不通问题现象Pod创建成功但无法跨节点通信解决方案检查Calico Pod是否正常运行检查节点防火墙是否放行Calico使用的端口默认4789和5473检查各节点路由表是否正确1.8.3 证书过期问题问题现象组件间通信失败日志显示证书过期解决方案提前规划证书有效期建议设置为10年定期检查证书有效期建立证书轮换机制1.9 性能优化建议etcd优化使用SSD磁盘适当增加etcd的heartbeat-interval和election-timeout定期进行etcd数据压缩kube-apiserver优化启用--enable-aggregator-routing适当增加--max-requests-inflight和--max-mutating-requests-inflight使用--target-ram-mb参数限制内存使用kubelet优化设置合理的--max-pods参数启用--serialize-image-pulls避免并发拉取镜像使用--image-gc-high-threshold和--image-gc-low-threshold控制镜像垃圾回收1.10 后续扩展方向高可用部署部署多个Master节点使用负载均衡器暴露apiserver部署多个etcd节点组成集群监控告警部署Prometheus Grafana监控集群配置关键指标告警使用kube-state-metrics收集集群状态日志收集部署EFKElasticsearchFluentdKibana栈配置日志轮转和归档策略设置日志级别和过滤规则存储方案部署Ceph或NFS提供持久化存储配置StorageClass实现动态供给使用Local PV优化本地存储性能通过这次二进制部署实践我深刻理解了Kubernetes各组件的工作机制和交互方式。虽然过程比使用kubeadm等工具复杂但对于想要真正掌握Kubernetes原理的工程师来说这种部署方式非常值得尝试。在实际生产环境中建议先通过二进制方式部署测试环境理解原理后再根据实际情况选择合适的部署工具。