CentOS 7下Java技术栈环境搭建与配置指南
1. 环境准备与基础组件安装在搭建这套技术栈之前我们需要先准备好基础环境。我选择CentOS 7作为操作系统平台因为它兼具稳定性和广泛的社区支持。以下是基础组件的安装步骤1.1 JDK安装与配置Java环境是整个技术栈的基础这里我选择JDK 8u271版本。安装过程需要注意几个关键点#!/bin/bash # 检查安装包是否存在 [ ! -f jdk-8u271-linux-x64.tar.gz ] echo JDK安装包不存在 exit # 解压并移动到标准目录 tar xf jdk-8u271-linux-x64.tar.gz mv jdk1.8.0_271 /usr/local/java # 设置环境变量 cat /etc/profile.d/java.sh EOF export JAVA_HOME/usr/local/java export PATH\$JAVA_HOME/bin:\$PATH EOF # 立即生效 source /etc/profile.d/java.sh java -version注意环境变量配置在/etc/profile.d/下比直接修改/etc/profile更规范便于维护和隔离不同组件的配置。1.2 Maven安装与配置Maven 3.5.4版本足够稳定适合大多数项目wget https://mirrors.bfsu.edu.cn/apache/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz tar xf apache-maven-3.5.4-bin.tar.gz mv apache-maven-3.5.4 /usr/local/maven3.5 cat /etc/profile.d/maven.sh EOF export MAVEN_HOME/usr/local/maven3.5 export PATH\$MAVEN_HOME/bin:\$PATH EOF source /etc/profile.d/maven.sh mvn -v在实际操作中我发现国内使用镜像源能显著提高依赖下载速度。可以在~/.m2/settings.xml中配置阿里云镜像mirror idaliyunmaven/id mirrorOf*/mirrorOf name阿里云公共仓库/name urlhttps://maven.aliyun.com/repository/public/url /mirror2. 数据库与中间件安装2.1 MongoDB安装与安全配置MongoDB 4.0.21版本在稳定性和功能之间取得了良好平衡。安装时特别注意安全配置#!/bin/bash PORT27017 Useradmin Passwdyour_strong_password wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel62-4.0.21.tgz tar -zxvf mongodb-linux-x86_64-rhel62-4.0.21.tgz mv mongodb-linux-x86_64-rhel62-4.0.21/ /usr/local/mongodb mkdir -p /usr/local/mongodb/{data,logs} touch /usr/local/mongodb/logs/mongo.log # 环境变量配置 cat /etc/profile.d/mongodb.sh EOF export MONGODB_HOME/usr/local/mongodb export PATH\$PATH:\$MONGODB_HOME/bin EOF source /etc/profile.d/mongodb.sh # 配置文件 cat /usr/local/mongodb/mongo.conf EOF dbpath/usr/local/mongodb/data logpath/usr/local/mongodb/logs/mongo.log logappendtrue journaltrue quiettrue port$PORT forktrue bind_ip0.0.0.0 authtrue EOF # Systemd服务配置 cat /usr/lib/systemd/system/mongodb.service EOF [Unit] Descriptionmongodb Afternetwork.target remote-fs.target nss-lookup.target [Service] Typeforking ExecStart/usr/local/mongodb/bin/mongod --config /usr/local/mongodb/mongo.conf ExecReload/bin/kill -s HUP \$MAINPID ExecStop/usr/local/mongodb/bin/mongod --shutdown --config /usr/local/mongodb/mongo.conf PrivateTmptrue Restartalways RestartSec1 [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable --now mongodb # 创建管理员账户 mongo --port $PORT admin --eval db.createUser({user:$User,pwd:$Passwd,roles:[userAdminAnyDatabase,root]})安全提示生产环境务必修改默认端口启用auth并设置复杂密码。bind_ip不要使用0.0.0.0应该限制为必要IP。2.2 Zookeeper集群部署Zookeeper 3.6.1版本对Kafka 2.6.0有良好支持。单节点部署如下#!/bin/bash HostIP$(hostname -I |awk {print $1}) [ ! -f apache-zookeeper-3.6.1-bin.tar.gz ] echo Zookeeper安装包缺失 exit tar -zxf apache-zookeeper-3.6.1-bin.tar.gz mv apache-zookeeper-3.6.1-bin /usr/local/zookeeper mkdir -p /usr/local/zookeeper/{data,datalog} # 配置文件修改 cd /usr/local/zookeeper/conf cp zoo_sample.cfg zoo.cfg sed -i s#dataDir/tmp/zookeeper#dataDir/usr/local/zookeeper/data# zoo.cfg echo dataLogDir/usr/local/zookeeper/datalog zoo.cfg echo server.1$HostIP:2888:3888 zoo.cfg echo 1 /usr/local/zookeeper/data/myid # 环境变量 cat /etc/profile.d/zookeeper.sh EOF export ZK_HOME/usr/local/zookeeper export PATH\$ZK_HOME/bin:\$PATH EOF source /etc/profile.d/zookeeper.sh # Systemd服务 cat /usr/lib/systemd/system/zookeeper.service EOF [Unit] Descriptionzookeeper.service Afternetwork.target ConditionPathExists/usr/local/zookeeper/conf/zoo.cfg [Service] Typeforking EnvironmentPATH/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/java/bin Userroot Grouproot ExecStart/usr/local/zookeeper/bin/zkServer.sh start ExecStop/usr/local/zookeeper/bin/zkServer.sh stop [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable --now zookeeper集群部署时需要在所有节点上配置相同的zoo.cfg包含所有server条目每个节点的myid文件内容要唯一1,2,3...防火墙开放2888和3888端口3. Kafka集群部署与管理3.1 Kafka核心安装Kafka 2.6.0版本与前述组件版本匹配良好#!/bin/bash HostIP$(hostname -I |awk {print $1}) PORT9092 [ ! -f kafka_2.13-2.6.0.tgz ] echo Kafka安装包缺失 exit tar -zxf kafka_2.13-2.6.0.tgz mv kafka_2.13-2.6.0 /usr/local/kafka mkdir -p /usr/local/kafka/kafka-logs # 配置文件调整 cd /usr/local/kafka/config cp server.properties server.properties.bak sed -i s#broker.id0#broker.id1# server.properties sed -i s#log.dirs/tmp/kafka-logs#log.dirs/usr/local/kafka/kafka-logs# server.properties echo listenersPLAINTEXT://:$PORT server.properties echo advertised.listenersPLAINTEXT://$HostIP:$PORT server.properties # 环境变量 cat /etc/profile.d/kafka.sh EOF export KAFKA_HOME/usr/local/kafka export PATH\$KAFKA_HOME/bin:\$PATH EOF source /etc/profile.d/kafka.sh # Systemd服务 cat /usr/lib/systemd/system/kafka.service EOF [Unit] DescriptionApache Kafka server (broker) Afternetwork.target zookeeper.service [Service] Typesimple EnvironmentPATH/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/java/bin Userroot Grouproot ExecStart/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties ExecStop/usr/local/kafka/bin/kafka-server-stop.sh [Install] WantedBymulti-user.target EOF systemctl daemon-reload systemctl enable --now kafka关键配置说明broker.id集群中必须唯一log.dirs建议使用高性能磁盘单独挂载listeners/advertised.listeners内外网访问配置zookeeper.connect集群模式下配置所有ZK节点3.2 Kafka基础操作创建topickafka-topics.sh --create --zookeeper localhost:2181 \ --replication-factor 1 --partitions 3 --topic test生产消息kafka-console-producer.sh --broker-list localhost:9092 --topic test消费消息kafka-console-consumer.sh --bootstrap-server localhost:9092 \ --topic test --from-beginning查看topic列表kafka-topics.sh --list --zookeeper localhost:21814. Kafka-Manager可视化监控4.1 安装与配置Kafka-Manager 2.0.0.2提供了友好的Web界面#!/bin/bash [ ! -f kafka-manager-2.0.0.2.zip ] echo Kafka-Manager安装包缺失 exit unzip kafka-manager-2.0.0.2.zip mv kafka-manager-2.0.0.2 /usr/local/kafka-manager # 修改配置 cd /usr/local/kafka-manager/conf sed -i s#kafka-manager-zookeeper:2181#localhost:2181# application.conf # 启动 nohup /usr/local/kafka-manager/bin/kafka-manager \ -Dconfig.file/usr/local/kafka-manager/conf/application.conf \ -Dhttp.port8090 -java-home /usr/local/java 4.2 使用技巧集群添加通过Web界面添加集群时填写正确的ZK地址和Kafka版本监控指标关注Under Replicated Partitions和Offline Partitions Count消费者监控可以查看消费滞后情况主题操作支持创建、删除、扩分区等操作生产环境建议启用basicAuthentication.enabledtrue并配置强密码5. 系统集成与优化5.1 组件连通性验证验证ZK与Kafka连接echo stat | nc localhost 2181 | grep Mode kafka-topics.sh --list --zookeeper localhost:2181验证MongoDB连接mongo --host localhost --port 27017 -u admin -p your_password --authenticationDatabase admin验证Kafka-Manager 访问http://your-server-ip:80905.2 性能优化建议Kafka优化调整num.io.threads和num.network.threads优化log.retention.hours和log.segment.bytes考虑使用SSD存储日志MongoDB优化配置WiredTiger缓存大小建立合适的索引启用journaling系统层面调整文件描述符限制优化swap和内存配置考虑使用cgroup限制资源5.3 常见问题排查Kafka启动失败检查ZK连接验证broker.id唯一性查看日志/usr/local/kafka/logs/server.log生产者无法发送消息检查advertised.listeners配置验证防火墙设置检查topic是否存在MongoDB连接问题检查auth是否启用验证bind_ip配置查看/usr/local/mongodb/logs/mongo.log这套技术栈整合了消息队列、分布式协调和文档数据库非常适合构建现代分布式系统。在实际部署中根据业务规模合理规划集群节点数量和资源配置是关键。
