4.2.2版本Connect to observer x [ERROR] OBD-1006: Failed to connect to oceanbase-ce

【 使用环境 】生产环境 or 测试环境
【 OB or 其他组件 】
【 使用版本 】4.2.2
【问题描述】清晰明确描述问题
root@localhost skoceanbase]# obd cluster start skoceanbase --wop
Get local repositories ok
Search plugins ok
Load cluster param plugin ok
Open ssh connection ok
Check before start observer ok
[WARN] OBD-1007: (192.168.199.21) The recommended number of open files is 655350 (Current value: 20000)
[WARN] OBD-1007: (192.168.199.21) The recommended number of max user processes is 655350 (Current value: 120000)
[WARN] OBD-1017: (192.168.199.21) The value of the “vm.max_map_count” must be within [327600, 1310720] (Current value: 65530, Recommended value: 655360)
[WARN] OBD-1017: (192.168.199.21) The value of the “fs.file-max” must be greater than 6573688 (Current value: 65535, Recommended value: 6573688)
[WARN] OBD-1007: (192.168.199.22) The recommended number of open files is 655350 (Current value: 20000)
[WARN] OBD-1007: (192.168.199.22) The recommended number of max user processes is 655350 (Current value: 120000)
[WARN] OBD-1017: (192.168.199.22) The value of the “vm.max_map_count” must be within [327600, 1310720] (Current value: 65530, Recommended value: 655360)
[WARN] OBD-1017: (192.168.199.22) The value of the “fs.file-max” must be greater than 6573688 (Current value: 65535, Recommended value: 6573688)
[WARN] OBD-2000: (192.168.199.22) not enough memory. (Free: 5268M, Need: 6144M)
[WARN] OBD-1007: (192.168.199.23) The recommended number of open files is 655350 (Current value: 20000)
[WARN] OBD-1007: (192.168.199.23) The recommended number of max user processes is 655350 (Current value: 120000)
[WARN] OBD-1017: (192.168.199.23) The value of the “vm.max_map_count” must be within [327600, 1310720] (Current value: 65530, Recommended value: 655360)
[WARN] OBD-1017: (192.168.199.23) The value of the “fs.file-max” must be greater than 6573688 (Current value: 65535, Recommended value: 6573688)
[WARN] OBD-2000: (192.168.199.23) not enough memory. (Free: 4959M, Need: 6144M)

Check before start obproxy ok
[WARN] OBD-4521: The config observer_sys_password in obproxy-ce did not take effect, please config it in oceanbase-ce

Check before start obagent ok
Check before start ocp-express ok
Start observer ok
observer program health check ok
obshell program health check ok
Connect to observer x
[ERROR] OBD-1006: Failed to connect to oceanbase-ce
See OceanBase分布式数据库-海量数据 笔笔算数 .
Trace ID: d9911012-4893-11ef-b1e6-525400028107
If you want to view detailed obd logs, please run: obd display-trace d9911012-4893-11ef-b1e6-525400028107

2 个赞

运行下如下命令贴出来,
obd display-trace d9911012-4893-11ef-b1e6-525400028107

检查下配置项 production_mode

2 个赞

我现在是测试环境。
3台
21

22

23

trace.txt (78.0 KB)

1 个赞

1.3节点检查下observer进程 ps -ef|grep observer 截图发下
2.是否修改过sys租户root用户的密码

2 个赞

服务器密码 应该没改过(数据库已经跑了一个多月了,系统开发了1/3了 然后前天7-21傍晚 然后突然就链接不上了)
一个多月前安装还好后 自己增加了一个用户
CREATE DATABASE 数据库名称 DEFAULT CHARACTER SET utf8mb4;
CREATE USER ‘用户名’ IDENTIFIED BY ‘密码’;
GRANT SELECT ON 数据库.* TO 用户名;

ps -ef|grep observer 结果如下
21

22

23

2 个赞

上传下config.yaml文件,类似路径/home/admin/.obd/cluster

另外登陆测试下密码是否正确:obclient -h192.168.199.21 -P2881 -uroot@sys -pUAMM.7tliF;:~| -Doceanbase -A

2 个赞

config.yaml.txt (2.6 KB)
tmp_config.yaml.txt (2.6 KB)

2 个赞

这里密码用单引号括一下 再试试
obclient -h192.168.199.21 -P2881 -uroot@sys -p‘UAMM.7tliF;:~|’ -Doceanbase -A

2 个赞


密码 密码用单引号括起来后,回车会提示输入密码 输入 UAMM.7tliF;:~| 后 话是
ERROR 8001 (08004): Server is initializing

2 个赞

-p后面不要留空格,
另外上传一下3个节点的observer.log

1 个赞

-p后面不带空格,密码带上单引号,如下图

3台服务器 observer.log 文件 都大于100M了,
我是不是把他们全部都删除掉 ,然后执行下 obd cluster reload skoceanbase (执行 obd cluster start skoceanbase 就提示我要执行 obd cluster reload skoceanbase)后 再将日志发给你了
21

22

23

可以先找一个小的压缩上传下,然后备份日志,删除日志,执行 obd cluster start skoceanbase

observer.log

提取码 1234

(压缩后 还超过10M 上传不上来,放到百度网盘上了)

已收到日志,分析中,同时执行如下命令,将报错截图发下
obd cluster stop skoceanbase
obd cluster start skoceanbase

分析日志,192.168.199.21上sys租户内存分配内存失败,实例异常,导致和其它observer的rpc通信失败,先检查一下192.168.199.21的内存使用情况,free -h

21这台

sys租户内存占用分析中

实际无占用内存较大的模块,主要是sys租户内存分配较小导致运行中出现问题,建议增加内存