1. 项目概述在Rocky Linux上通过二进制方式部署Kubernetes 1.33集群是当前企业级容器编排平台搭建的主流方案之一。不同于使用kubeadm等工具自动化安装二进制部署要求管理员手动配置每个核心组件包括etcd、kube-apiserver、kube-controller-manager、kube-scheduler等。这种方式虽然步骤繁琐但能让你彻底掌握Kubernetes的底层运行机制特别适合需要深度定制化配置的生产环境。我最近在客户现场实施了一套基于Rocky Linux 8.6的Kubernetes 1.33集群过程中积累了不少实战经验。相比常见的CentOS/RHEL系发行版Rocky Linux作为CentOS的替代品在软件包管理和系统配置上有一些独特之处需要特别注意。本文将详细记录从零开始搭建的全过程包括组件编译、证书生成、服务配置等关键环节。2. 环境准备与规划2.1 硬件资源配置建议对于生产环境建议至少准备三台物理机或虚拟机作为集群节点控制平面节点4核CPU/8GB内存/100GB磁盘建议奇数台如3台实现高可用Worker节点根据工作负载需求调整建议8核CPU/16GB内存/200GB磁盘起步网络要求节点间延迟1ms万兆网络最佳重要提示所有节点必须保证时间同步chrony/NTP否则证书验证会失败2.2 操作系统配置以Rocky Linux 8.6为例基础配置步骤如下# 关闭SELinux生产环境需根据安全策略调整 setenforce 0 sed -i s/SELINUXenforcing/SELINUXpermissive/ /etc/selinux/config # 关闭防火墙或配置放行规则 systemctl stop firewalld systemctl disable firewalld # 加载内核模块 cat /etc/modules-load.d/k8s.conf EOF br_netfilter ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack EOF # 加载模块 modprobe -- br_netfilter ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack # 配置内核参数 cat /etc/sysctl.d/k8s.conf EOF net.bridge.bridge-nf-call-ip6tables 1 net.bridge.bridge-nf-call-iptables 1 net.ipv4.ip_forward 1 EOF sysctl --system2.3 依赖软件安装dnf install -y conntrack-tools socat ebtables ipset curl wget tar3. 核心组件部署3.1 证书体系搭建Kubernetes重度依赖TLS证书进行组件间通信认证。我们使用cfssl工具链生成自签名证书# 下载cfssl工具 wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O /usr/local/bin/cfssl wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O /usr/local/bin/cfssljson chmod x /usr/local/bin/cfssl* # 创建CA配置文件 cat ca-config.json EOF { signing: { default: { expiry: 87600h }, profiles: { kubernetes: { usages: [signing, key encipherment, server auth, client auth], expiry: 87600h } } } } EOF # 生成CA证书 cat ca-csr.json EOF { CN: Kubernetes, key: { algo: rsa, size: 2048 }, names: [ { C: CN, L: Beijing, O: Kubernetes, OU: CA, ST: Beijing } ] } EOF cfssl gencert -initca ca-csr.json | cfssljson -bare ca3.2 Etcd集群部署Etcd作为Kubernetes的后端存储需要首先部署# 下载etcd二进制包 ETCD_VERv3.5.4 wget https://github.com/etcd-io/etcd/releases/download/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz tar -xvf etcd-${ETCD_VER}-linux-amd64.tar.gz mv etcd-${ETCD_VER}-linux-amd64/etcd* /usr/local/bin/ # 创建systemd服务文件 cat /etc/systemd/system/etcd.service EOF [Unit] DescriptionEtcd Server Afternetwork.target [Service] Typenotify ExecStart/usr/local/bin/etcd \\ --nameetcd1 \\ --data-dir/var/lib/etcd \\ --initial-advertise-peer-urlshttps://192.168.1.101:2380 \\ --listen-peer-urlshttps://192.168.1.101:2380 \\ --listen-client-urlshttps://192.168.1.101:2379,https://127.0.0.1:2379 \\ --advertise-client-urlshttps://192.168.1.101:2379 \\ --initial-clusteretcd1https://192.168.1.101:2380,etcd2https://192.168.1.102:2380,etcd3https://192.168.1.103:2380 \\ --initial-cluster-tokenetcd-cluster \\ --initial-cluster-statenew \\ --client-cert-auth \\ --trusted-ca-file/etc/kubernetes/pki/ca.crt \\ --cert-file/etc/kubernetes/pki/etcd/server.crt \\ --key-file/etc/kubernetes/pki/etcd/server.key \\ --peer-client-cert-auth \\ --peer-trusted-ca-file/etc/kubernetes/pki/ca.crt \\ --peer-cert-file/etc/kubernetes/pki/etcd/peer.crt \\ --peer-key-file/etc/kubernetes/pki/etcd/peer.key Restarton-failure RestartSec5 [Install] WantedBymulti-user.target EOF3.3 Kubernetes控制平面部署3.3.1 组件下载与安装# 下载Kubernetes二进制文件 K8S_VERv1.33.0 wget https://dl.k8s.io/${K8S_VER}/kubernetes-server-linux-amd64.tar.gz tar -xvf kubernetes-server-linux-amd64.tar.gz cd kubernetes/server/bin cp kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/3.3.2 API Server配置cat /etc/systemd/system/kube-apiserver.service EOF [Unit] DescriptionKubernetes API Server Documentationhttps://github.com/kubernetes/kubernetes Afteretcd.service [Service] ExecStart/usr/local/bin/kube-apiserver \\ --advertise-address192.168.1.101 \\ --allow-privilegedtrue \\ --apiserver-count3 \\ --audit-log-maxage30 \\ --audit-log-maxbackup3 \\ --audit-log-maxsize100 \\ --audit-log-path/var/log/audit.log \\ --authorization-modeNode,RBAC \\ --bind-address0.0.0.0 \\ --client-ca-file/etc/kubernetes/pki/ca.crt \\ --enable-admission-pluginsNodeRestriction \\ --enable-bootstrap-token-authtrue \\ --etcd-cafile/etc/kubernetes/pki/ca.crt \\ --etcd-certfile/etc/kubernetes/pki/apiserver-etcd-client.crt \\ --etcd-keyfile/etc/kubernetes/pki/apiserver-etcd-client.key \\ --etcd-servershttps://192.168.1.101:2379,https://192.168.1.102:2379,https://192.168.1.103:2379 \\ --kubelet-client-certificate/etc/kubernetes/pki/apiserver-kubelet-client.crt \\ --kubelet-client-key/etc/kubernetes/pki/apiserver-kubelet-client.key \\ --service-account-key-file/etc/kubernetes/pki/sa.pub \\ --service-account-signing-key-file/etc/kubernetes/pki/sa.key \\ --service-account-issuerhttps://kubernetes.default.svc.cluster.local \\ --service-cluster-ip-range10.96.0.0/12 \\ --tls-cert-file/etc/kubernetes/pki/apiserver.crt \\ --tls-private-key-file/etc/kubernetes/pki/apiserver.key \\ --requestheader-client-ca-file/etc/kubernetes/pki/front-proxy-ca.crt \\ --proxy-client-cert-file/etc/kubernetes/pki/front-proxy-client.crt \\ --proxy-client-key-file/etc/kubernetes/pki/front-proxy-client.key \\ --requestheader-allowed-namesfront-proxy-client \\ --requestheader-extra-headers-prefixX-Remote-Extra- \\ --requestheader-group-headersX-Remote-Group \\ --requestheader-username-headersX-Remote-User \\ --secure-port6443 Restarton-failure RestartSec5 [Install] WantedBymulti-user.target EOF4. 网络与插件配置4.1 CNI插件选择与部署推荐使用Calico作为网络插件curl https://docs.projectcalico.org/manifests/calico.yaml -O kubectl apply -f calico.yaml4.2 CoreDNS配置# 下载CoreDNS配置文件 wget https://github.com/coredns/deployment/blob/master/kubernetes/coredns.yaml.sed # 修改配置后部署 kubectl apply -f coredns.yaml5. 节点加入与验证5.1 Worker节点准备在每个Worker节点上# 安装kubelet kube-proxy cp kubelet kube-proxy /usr/local/bin/ # 配置kubelet cat /etc/systemd/system/kubelet.service EOF [Unit] DescriptionKubernetes Kubelet Afterdocker.service [Service] ExecStart/usr/local/bin/kubelet \\ --bootstrap-kubeconfig/etc/kubernetes/bootstrap-kubelet.conf \\ --kubeconfig/etc/kubernetes/kubelet.conf \\ --pod-manifest-path/etc/kubernetes/manifests \\ --allow-privilegedtrue \\ --network-plugincni \\ --cni-conf-dir/etc/cni/net.d \\ --cni-bin-dir/opt/cni/bin \\ --cluster-dns10.96.0.10 \\ --cluster-domaincluster.local \\ --fail-swap-onfalse \\ --runtime-cgroups/systemd/system.slice \\ --kubelet-cgroups/systemd/system.slice Restarton-failure RestartSec5 [Install] WantedBymulti-user.target EOF5.2 集群验证kubectl get nodes kubectl get pods --all-namespaces6. 常见问题排查6.1 证书相关错误症状x509: certificate has expired or is not yet valid解决方案检查所有节点时间同步重新生成证书并设置更长有效期6.2 Etcd集群无法启动症状etcdserver: unhealthy cluster解决方案检查etcd节点间网络连通性验证证书配置是否正确清理数据目录后重新初始化6.3 网络插件问题症状Pods处于ContainerCreating状态解决方案检查CNI二进制文件权限验证网络插件日志确认内核模块加载正确7. 性能调优建议Etcd优化单独SSD磁盘用于etcd数据目录调整心跳间隔和选举超时定期进行碎片整理API Server优化启用API优先级和公平性调整--max-requests-inflight参数使用本地SSD缓存kubelet优化设置合理的--max-pods参数启用Pod生命周期事件生成器调整镜像垃圾回收阈值在Rocky Linux上完成Kubernetes二进制部署后建议运行集群一致性测试工具如sonobuoy进行完整验证。对于生产环境还需要考虑日志收集、监控告警、备份恢复等配套系统的搭建。