物理机部署opengeminiopengemini物理机部署由于k8s部署后meta在重启集群后会丢失登录信息,所以最后还是切换到了使用物理机部署,目前部署的是3台机器,配置免密登录## 每个机子都执行一次ssh-keygen-trsa-b4096-Cj_janle163.com## 每个机子都执行一次到其他机子ssh-copy-id root172.16.104.7机器部署情况机器部署文件的路径在 /var/lib/openGemini/deploy,数据文件目录/var/openGemini/data,目前每个机子都分配了5T存储对应的机器部署文件部署配置启动脚本部署监控172.16.104.7/var/lib/openGemini/deploy//var/lib/openGemini/deploy/usr/config/openGemini.conf/var/lib/openGemini/deploy/usr/bin/ts-meta.sh ts-store2.shts-sql3.sh172.16.104.8/var/lib/openGemini/deploy//var/lib/openGemini/deploy/usr/config/openGemini1.conf/var/lib/openGemini/deploy/usr/bin/ts-meta.sh ts-store2.shts-sql3.sh172.16.104.9/var/lib/openGemini/deploy//var/lib/openGemini/deploy/usr/config/openGemini2.conf/var/lib/openGemini/deploy/usr/bin/ts-meta.sh ts-store2.shts-sql3.sh部署路径/var/lib/openGemini/deploy/usr/bin配置路径/var/lib/openGemini/deploy/usr/configopenGemini.conf的配置[common]auth-enabledtruemeta-join[172.16.104.7:8092,172.16.104.8:8092,172.16.104.9:8092][meta]# 需要绑定当前集群的地址bind-address172.16.104.9:8088http-bind-address172.16.104.9:8091rpc-bind-address172.16.104.9:8092# 建议修改目录dir/var/openGemini/data/meta[http]auth-enabledtruebind-address172.16.104.9:8086[data]store-ingest-addr172.16.104.9:8400store-select-addr172.16.104.9:8401# 建议修改目录store-data-dir/var/openGemini/data/datastore-wal-dir/var/openGemini/data/walstore-meta-dir/var/openGemini/data/meta[logging]# 建议修改目录path/var/lib/openGemini/logs[gossip]bind-address172.16.104.9store-bind-port8011meta-bind-port8010members[172.16.104.7:8010,172.16.104.8:8010,172.16.104.9:8010]数据文件配置启动脚本– meta对应的启动脚本#!/bin/bashnohup/var/lib/openGemini/deploy/usr/bin/ts-meta--config/var/lib/openGemini/deploy/usr/config/openGemini.conf-pidfilemeta.pidmeta_extra.log21– store对应的启动脚本#!/bin/bashnohup/var/lib/openGemini/deploy/usr/bin/ts-store--config/var/lib/openGemini/deploy/usr/config/openGemini.conf-pidfilestore.pidstore_extra.log21– sql对应的启动脚本#!/bin/bashnohup/var/lib/openGemini/deploy/usr/bin/ts-sql--config/var/lib/openGemini/deploy/usr/config/openGemini.conf-pidfilesql.pidsql_extra.log21注意事项保证对应的数据库的相关的操作数据硬盘足够,这样不至于有别的问题.保证网络带宽是足够的情况遇到问题问题1现场日志可以登录集群,但是集群查询数据时候查询不到数据2024-10-13T23:55:51.478Z panic /home/runner/work/openGemini/openGemini/lib/util/lifted/vm/fs/fs_unix.go:30 FATAL: cannot determinefreedisk space on/var/openGemini/data/data/data/gps_timedb/0/rp180d/index/994_1728435600000000000_1728460800000000000/mergeset:open/var/openGemini/data/data/data/gps_timedb/0/rp180d/index/994_1728435600000000000_1728460800000000000/mergeset: too manyopenfiles panic: FATAL: cannot determinefreedisk space on/var/openGemini/data/data/data/gps_timedb/0/rp180d/index/994_1728435600000000000_1728460800000000000/mergeset:open/var/openGemini/data/data/data/gps_timedb/0/rp180d/index/994_1728435600000000000_1728460800000000000/mergeset: too manyopenfiles排查过程检查集群状态./ts-cli--host172.16.104.7--port8086--usernamejanle--passwordJanle5opengemini show cluster三台机子都是failed解决办法# 小型部署16384 中型部署32768 大型部署65536# 执行以下命令显示1024,太小ulimit-n#设置为655350setulimit655350#limit.conf 添加以下配置vi/etc/security/limit.conf * soft nofile655350* hard nofile655350问题2现场日志ts-store启动失败,一直在集群显示上显示的faildcatstore_extra.log parse config: parse config: Near line11(last key parsedhttp): expected a t解决办法检查对应的配置文件,是否有不合法的字符或者配置有问题k8s部署opengeminik8s安装部署 ## 创建storeclass和namespace kubectl apply -f opengemini-sc.yaml ## 创建存储 kubectl apply -f opengemini-stroepv.yaml #创建存储的pvpvc kubectl apply -f opengemini-metapv.yaml #创meta的pvpvc ## 创建网络服务 kubectl apply -f opengemini-service.yaml #创meta的pvpvc ## 创建有状态部署 部署按照启动顺序执行 kubectl apply -f opengemini-StatefulSet-meta.yaml #创meta的pod kubectl apply -f opengemini-StatefulSet-store.yaml #创meta的pod kubectl apply -f opengemini-StatefulSet-sql.yaml #创meta的pod 资源规划操作使用用户安全如使用openGemini集群修改配置文件openGemini.conf[内网就不需要了][http] auth-enabled true # 内网下边不需要配置 #https-enabled true #https-certificate path/to/certificate.crt #https-private-key path/to/certificate.key配置完成后重启初始状态创建管理员账号curl -i -XPOST http://10.233.82.58:8086/query -k --insecure --data-urlencode qCREATE USER janle WITH PASSWORD Janle5opengemini WITH ALL PRIVILEGES账户授权操作## auth 登录后showusers;## 创建用户CREATEUSERuser0 WITH PASSWORDyour_pwdshow databases;## 授权对用户的读写和全部权限GRANT[READ,WRITE,ALL]ONdatabase_nameTOusernameGRANT ALL ONNOAA_water_databaseTOuser0## 展示授权SHOW GRANTS FORusernameREVOKE ALL ONNOAA_water_databaseFROMuser0REVOKE[READ,WRITE,ALL]ONdatabase_nameFROMusernameSET PASSWORD FORusernamepasswordDROPUSERusernameSchema查询## 创建数据库CREATEDATABASEdatabase_name[WITH[DURATIONduration][REPLICATIONn][SHARD DURATIONduration][INDEXDURATIONduration][NAMEretention-policy-name]]-- 样例CREATEDATABASEmy_timeseries_dbWITHDURATION10d-- 数据库中的数据保留期限为30天-- REPLICATION 2 -- 设置数据副本数量为2 这个应该还没有上线SHARD DURATION7d-- 每个分片的数据保留期限为7天INDEXDURATION1d-- 索引的保留期限为1天NAMEdefault_rp-- 设置默认的保留策略名称为default_rp;## 查看数据库showdatabasesusedatabases## 当前database下的measurementsshowmeasurements## 了解下业务的模型,查看tag,field,部分series内容showtagkeysshowfieldkeysshowseries## 了解当前服务有多少时间线,一共多少台机器(ip的value个数)showseries cardinalityshowtagvaluescardinalitywithkey## 查看数据库的保留策略SHOWRETENTION POLICIES[ONdatabase_name]SHOWRETENTION POLICIESONNOAA_water_database聚合查询## 查询某某台机子是否存在异常情况?聚合分组查询SELECTfunction(fieldkey)FROM_clauseWHEREtime_rangeGROUPBYtime(time_interval),[tagkey][fill(filloption)]## 具体的聚合函数有 min,max,sum,count,mean,first,lastselectcount(field)from_clauseWHEREtime_rangeGROUPBYtime(time_interval),[tagkey][fill(filloption)]## 子查询## 多表查询非聚合查询其他