1. 项目概述在容器化技术普及的今天企业级镜像仓库已成为DevOps流水线中不可或缺的基础设施。Harbor作为CNCF毕业项目是目前最成熟的开源企业级镜像仓库解决方案之一。本文将基于Rocky Linux 8/9操作系统详细讲解如何构建一个生产级高可用的Harbor集群。提示本文方案适用于日均镜像推送量超过5000次的中大型企业环境所有配置参数均经过实际生产验证。2. 架构设计与组件选型2.1 高可用架构设计我们采用经典的双活共享存储架构前端2个Harbor节点通过Nginx实现负载均衡 后端PostgreSQL集群3节点 Redis哨兵集群3节点 存储MinIO集群4节点纠删码模式这种设计具有以下优势任意单节点故障不影响服务可用性读写性能随节点数量线性扩展存储层采用纠删码技术磁盘利用率高达80%2.2 关键组件版本选择组件版本选择理由Rocky Linux8.7/9.1RHEL兼容性最佳Harborv2.7.2长期支持版本PostgreSQL14.5官方推荐版本Redis6.2.7稳定性验证MinIORELEASE.2023-08-29T23-07-06Z支持最新纠删码算法3. 基础环境准备3.1 操作系统配置首先在所有节点执行# 关闭SELinux sudo sed -i s/SELINUXenforcing/SELINUXdisabled/g /etc/selinux/config sudo setenforce 0 # 配置防火墙 sudo firewall-cmd --permanent --add-port80/tcp sudo firewall-cmd --permanent --add-port443/tcp sudo firewall-cmd --permanent --add-port5432/tcp sudo firewall-cmd --permanent --add-port6379/tcp sudo firewall-cmd --reload # 优化内核参数 cat EOF | sudo tee /etc/sysctl.d/99-harbor.conf net.ipv4.ip_local_port_range 1024 65000 net.core.somaxconn 10240 net.ipv4.tcp_max_syn_backlog 20480 vm.max_map_count262144 EOF sudo sysctl -p /etc/sysctl.d/99-harbor.conf3.2 存储层部署MinIO以4节点为例每个节点执行wget https://dl.min.io/server/minio/release/linux-amd64/minio chmod x minio sudo mv minio /usr/local/bin/ # 创建数据目录 sudo mkdir -p /data/minio{1..4}配置systemd服务/etc/systemd/system/minio.service[Unit] DescriptionMinIO Afternetwork.target [Service] Userminio Groupminio EnvironmentMINIO_ROOT_USERadmin EnvironmentMINIO_ROOT_PASSWORDYourStrongPassword ExecStart/usr/local/bin/minio server http://node{1..4}/data/minio{1..4} \ --console-address :9001 Restartalways [Install] WantedBymulti-user.target启动集群sudo systemctl daemon-reload sudo systemctl enable --now minio4. Harbor集群部署4.1 数据库集群配置使用Patroni管理PostgreSQL集群sudo dnf install -y postgresql14 postgresql14-server patroni配置Patroni/etc/patroni.ymlscope: harbor-pg namespace: /service/ name: pg-node1 restapi: listen: 0.0.0.0:8008 connect_address: 192.168.1.101:8008 etcd: hosts: [etcd1:2379,etcd2:2379,etcd3:2379] bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 postgresql: use_pg_rewind: true parameters: max_connections: 500 shared_buffers: 4GB wal_level: logical postgresql: listen: 0.0.0.0:5432 connect_address: 192.0.2.101:5432 data_dir: /var/lib/pgsql/14/data bin_dir: /usr/pgsql-14/bin pgpass: /tmp/pgpass authentication: replication: username: replicator password: ReplicaPass123 superuser: username: postgres password: PostgresPass1234.2 Harbor核心安装下载并解压Harbor离线包wget https://github.com/goharbor/harbor/releases/download/v2.7.2/harbor-offline-installer-v2.7.2.tgz tar xvf harbor-offline-installer-v2.7.2.tgz cd harbor配置harbor.ymlhostname: harbor.yourdomain.com http: port: 80 https: port: 443 certificate: /etc/ssl/certs/harbor.crt private_key: /etc/ssl/certs/harbor.key external_url: https://harbor.yourdomain.com harbor_admin_password: Harbor12345 database: password: pgpass123 max_idle_conns: 50 max_open_conns: 100 redis: password: redispass123 storage_service: s3: accesskey: minioadmin secretkey: YourStrongPassword region: us-east-1 bucket: harbor-registry secure: true endpoint: https://minio.yourdomain.com执行安装sudo ./install.sh --with-trivy --with-chartmuseum5. 高可用配置5.1 负载均衡设置使用Nginx实现流量分发upstream harbor { least_conn; server harbor-node1:80 max_fails3 fail_timeout30s; server harbor-node2:80 max_fails3 fail_timeout30s; keepalive 32; } server { listen 80; server_name harbor.yourdomain.com; location / { proxy_pass http://harbor; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_http_version 1.1; proxy_set_header Connection ; } }5.2 数据同步方案使用Harbor的复制策略实现跨集群同步在管理界面创建复制规则选择基于事件的触发模式设置过滤条件可选启用覆盖镜像选项注意大规模集群建议设置带宽限制避免同步流量影响生产业务6. 运维与监控6.1 健康检查端点组件检查端点预期返回值Harbor/api/v2.0/health{status:healthy}PostgreSQL/patroniHTTP 200RedisPINGPONGMinIO/minio/health/liveHTTP 2006.2 关键监控指标配置Prometheus抓取规则- job_name: harbor metrics_path: /metrics static_configs: - targets: [harbor-node1:8080, harbor-node2:8080] - job_name: postgresql static_configs: - targets: [pg-node1:9187, pg-node2:9187, pg-node3:9187] - job_name: minio metrics_path: /minio/v2/metrics/cluster static_configs: - targets: [minio1:9000, minio2:9000]7. 故障处理实录7.1 常见问题排查问题1镜像推送失败报错blob upload invalid检查MinIO集群状态验证存储桶权限查看Harbor核心日志journalctl -u harbor -f问题2数据库连接池耗尽调整PostgreSQL最大连接数优化Harbor连接池配置database: max_idle_conns: 30 max_open_conns: 100问题3Redis响应延迟高检查哨兵选举状态增加Redis实例内存启用持久化日志7.2 性能优化参数在harbor.yml中添加metric: enabled: true port: 8080 jobservice: max_job_workers: 50 registry: middleware: storage: delete: enabled: true cache: blobdescriptor: redis8. 升级与维护8.1 滚动升级步骤从负载均衡中摘除第一个节点停止服务sudo docker-compose down备份数据sudo cp -r /data/registry /backups升级软件包启动服务sudo docker-compose up -d验证服务健康状态重复上述步骤升级其他节点8.2 数据备份策略每日全量备份脚本示例#!/bin/bash DATE$(date %Y%m%d) # 备份数据库 pg_dump -h pg-node1 -U postgres harbor /backups/harbor-db-$DATE.sql # 备份配置文件 tar czvf /backups/harbor-config-$DATE.tgz /harbor/harbor.yml /etc/ssl/certs/ # 同步到异地 rclone copy /backups remote:backups/harbor/9. 安全加固措施9.1 网络隔离方案建议的网络分区前端负载均衡DMZ区Harbor应用节点应用区数据库集群数据区MinIO存储存储区9.2 证书管理使用certbot自动续期sudo certbot certonly --standalone -d harbor.yourdomain.com sudo cp /etc/letsencrypt/live/harbor.yourdomain.com/{fullchain.pem,privkey.pem} /etc/ssl/certs/ sudo chown -R harbor:harbor /etc/ssl/certs/10. 扩展与集成10.1 与CI/CD流水线集成在Jenkins中配置pipeline { environment { HARBOR_CRED credentials(harbor-creds) } stages { stage(Build) { steps { sh docker build -t harbor.yourdomain.com/myapp:${BUILD_NUMBER} . } } stage(Push) { steps { sh docker login -u $HARBOR_CRED_USR -p $HARBOR_CRED_PSW harbor.yourdomain.com docker push harbor.yourdomain.com/myapp:${BUILD_NUMBER} } } } }10.2 镜像扫描集成配置Trivy定时扫描trivy: ignore_unfixed: true skip_update: false security_check: vuln severity: high,critical scanners: - vuln - config cache_dir: /tmp/trivy在实际运维中我们发现Rocky Linux的稳定性与RHEL完全一致但需要特别注意SELinux策略的调整。对于超过500节点的Kubernetes集群建议将Harbor的Redis后端升级为Redis Cluster模式以应对高并发场景下的性能需求。