【 使用环境 】 测试环境
【 OB or 其他组件 】命令行部署
【 使用版本 】 OceanBase社区版一键安装包 (OceanBase All in One) V4.1.0_CE_BP2 X86
【问题描述】使用obd cluster start obtest启动数据库失败
[ERROR] oceanbase-ce start failed
【复现路径】根据文档的单机部署OceanBase时，按照文档进行操作，执行到启动时，出现错误
observer program health check x
之后按照文档中的错误代码修改了内存大小，还是出现该错误
【问题现象及影响】
数据库启动失败
【附件】
[2023-07-26 14:08:32.003] [INFO] Open ssh connection
[2023-07-26 14:08:32.136] [INFO] Load cluster param plugin
[2023-07-26 14:08:32.137] [DEBUG] - Searching param plugin for components …
[2023-07-26 14:08:32.137] [DEBUG] - Search param plugin for oceanbase-ce
[2023-07-26 14:08:32.137] [DEBUG] - Found for oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.1
[2023-07-26 14:08:32.137] [DEBUG] - Applying oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.1-102000042023061314.el7-d03fafa6fa8ceb0636e4db05b5b5f6c3ac2256a3
[2023-07-26 14:08:32.590] [DEBUG] - Call oceanbase-ce-py_script_start_check-4.0.0.0 for oceanbase-ce-4.1.0.1-102000042023061314.el7-d03fafa6fa8ceb0636e4db05b5b5f6c3ac2256a3
[2023-07-26 14:08:32.590] [DEBUG] - import start_check
[2023-07-26 14:08:32.611] [DEBUG] - add start_check ref count to 1
[2023-07-26 14:08:32.614] [INFO] Check before start observer
[2023-07-26 14:08:32.617] [DEBUG] – root@192.168.111.136 execute: ls /data/clog/tenant_1/
[2023-07-26 14:08:32.636] [DEBUG] – exited code 2, error output:
[2023-07-26 14:08:32.636] [DEBUG] ls: 无法访问/data/clog/tenant_1/: 没有那个文件或目录
[2023-07-26 14:08:32.636] [DEBUG]
[2023-07-26 14:08:32.636] [DEBUG] – root@192.168.111.136 execute: cat /root/observer/run/observer.pid
[2023-07-26 14:08:32.693] [DEBUG] – exited code 1, error output:
[2023-07-26 14:08:32.693] [DEBUG] cat: /root/observer/run/observer.pid: 没有那个文件或目录
[2023-07-26 14:08:32.693] [DEBUG]
[2023-07-26 14:08:32.694] [DEBUG] – 192.168.111.136 port check
[2023-07-26 14:08:32.694] [DEBUG] – root@192.168.111.136 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B41’ | awk -F’ ’ ‘{print $2}’ | uniq
[2023-07-26 14:08:32.757] [DEBUG] – exited code 0
[2023-07-26 14:08:32.757] [DEBUG] – root@192.168.111.136 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B42’ | awk -F’ ’ ‘{print $2}’ | uniq
[2023-07-26 14:08:32.843] [DEBUG] – exited code 0
[2023-07-26 14:08:32.844] [DEBUG] – root@192.168.111.136 execute: ls /data/sstable/block_file
[2023-07-26 14:08:32.910] [DEBUG] – exited code 2, error output:
[2023-07-26 14:08:32.910] [DEBUG] ls: 无法访问/data/sstable/block_file: 没有那个文件或目录
[2023-07-26 14:08:32.910] [DEBUG]
[2023-07-26 14:08:32.911] [DEBUG] – root@192.168.111.136 execute: grep -e ‘^ *em1:’ /proc/net/dev
[2023-07-26 14:08:32.988] [DEBUG] – exited code 0
[2023-07-26 14:08:32.989] [DEBUG] – root@192.168.111.136 execute: cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr
[2023-07-26 14:08:33.067] [DEBUG] – exited code 0
[2023-07-26 14:08:33.067] [DEBUG] – root@192.168.111.136 execute: ulimit -a
[2023-07-26 14:08:33.143] [DEBUG] – exited code 0
[2023-07-26 14:08:33.145] [DEBUG] – root@192.168.111.136 execute: cat /proc/meminfo
[2023-07-26 14:08:33.220] [DEBUG] – exited code 0
[2023-07-26 14:08:33.222] [DEBUG] – root@192.168.111.136 execute: df --block-size=1024
[2023-07-26 14:08:33.308] [DEBUG] – exited code 0
[2023-07-26 14:08:33.310] [DEBUG] – get disk info for path /dev, total: 16656379904 avail: 16656379904
[2023-07-26 14:08:33.310] [DEBUG] – get disk info for path /dev/shm, total: 16669224960 avail: 16669204480
[2023-07-26 14:08:33.310] [DEBUG] – get disk info for path /run, total: 16669224960 avail: 16550539264
[2023-07-26 14:08:33.310] [DEBUG] – get disk info for path /sys/fs/cgroup, total: 16669224960 avail: 16669224960
[2023-07-26 14:08:33.311] [DEBUG] – get disk info for path /, total: 998427688960 avail: 777626509312
[2023-07-26 14:08:33.311] [DEBUG] – get disk info for path /boot, total: 1063256064 avail: 870584320
[2023-07-26 14:08:33.311] [DEBUG] – get disk info for path /boot/efi, total: 209489920 avail: 197894144
[2023-07-26 14:08:33.311] [DEBUG] – get disk info for path /run/user/0, total: 3333844992 avail: 3333844992
[2023-07-26 14:08:33.312] [DEBUG] – root@192.168.111.136 execute: df --block-size=1024 /redo/clog
[2023-07-26 14:08:33.386] [DEBUG] – exited code 0
[2023-07-26 14:08:33.387] [DEBUG] – get disk info for path /, total: 998427688960 avail: 777626509312
[2023-07-26 14:08:33.388] [DEBUG] – root@192.168.111.136 execute: df --block-size=1024 /data
[2023-07-26 14:08:33.466] [DEBUG] – exited code 0
[2023-07-26 14:08:33.467] [DEBUG] – get disk info for path /, total: 998427688960 avail: 777626509312
[2023-07-26 14:08:33.467] [DEBUG] – disk: {’/dev’: {‘total’: 16656379904, ‘avail’: 16656379904, ‘need’: 0}, ‘/dev/shm’: {‘total’: 16669224960, ‘avail’: 16669204480, ‘need’: 0}, ‘/run’: {‘total’: 16669224960, ‘avail’: 16550539264, ‘need’: 0}, ‘/sys/fs/cgroup’: {‘total’: 16669224960, ‘avail’: 16669224960, ‘need’: 0}, ‘/’: {‘total’: 998427688960, ‘avail’: 777626509312, ‘need’: 0}, ‘/boot’: {‘total’: 1063256064, ‘avail’: 870584320, ‘need’: 0}, ‘/boot/efi’: {‘total’: 209489920, ‘avail’: 197894144, ‘need’: 0}, ‘/run/user/0’: {‘total’: 3333844992, ‘avail’: 3333844992, ‘need’: 0}}
[2023-07-26 14:08:33.467] [WARNING] OBD-1012: (192.168.111.136) clog and data use the same disk (/)
[2023-07-26 14:08:33.468] [DEBUG] – root@192.168.111.136 execute: date +%s%N
[2023-07-26 14:08:33.529] [DEBUG] – exited code 0
[2023-07-26 14:08:33.529] [DEBUG] – 192.168.111.136 time delta -0.95068359375
[2023-07-26 14:08:33.532] [INFO] [WARN] OBD-1012: (192.168.111.136) clog and data use the same disk (/)
[2023-07-26 14:08:33.532] [INFO]
[2023-07-26 14:08:33.532] [DEBUG] - sub start_check ref count to 0
[2023-07-26 14:08:33.532] [DEBUG] - export start_check
[2023-07-26 14:08:33.532] [DEBUG] - Call oceanbase-ce-py_script_start-4.0.0.0 for oceanbase-ce-4.1.0.1-102000042023061314.el7-d03fafa6fa8ceb0636e4db05b5b5f6c3ac2256a3
[2023-07-26 14:08:33.532] [DEBUG] - import start
[2023-07-26 14:08:33.533] [DEBUG] - add start ref count to 1
[2023-07-26 14:08:33.533] [INFO] Start observer
[2023-07-26 14:08:33.533] [DEBUG] – root@192.168.111.136 execute: ls /data/clog/tenant_1/
[2023-07-26 14:08:33.607] [DEBUG] – exited code 2, error output:
[2023-07-26 14:08:33.607] [DEBUG] ls: 无法访问/data/clog/tenant_1/: 没有那个文件或目录
[2023-07-26 14:08:33.607] [DEBUG]
[2023-07-26 14:08:33.608] [DEBUG] – root@192.168.111.136 execute: cat /root/observer/run/observer.pid
[2023-07-26 14:08:33.684] [DEBUG] – exited code 1, error output:
[2023-07-26 14:08:33.685] [DEBUG] cat: /root/observer/run/observer.pid: 没有那个文件或目录
[2023-07-26 14:08:33.685] [DEBUG]
[2023-07-26 14:08:33.685] [DEBUG] – 192.168.111.136 start command construction
[2023-07-26 14:08:33.686] [DEBUG] – starting 192.168.111.136 observer
[2023-07-26 14:08:33.687] [DEBUG] – root@192.168.111.136 set env LD_LIBRARY_PATH to ‘/root/observer/lib:’
[2023-07-26 14:08:33.688] [DEBUG] – root@192.168.111.136 execute: cd /root/observer; /root/observer/bin/observer -r ‘192.168.111.136:2882:2881’ -p 2881 -P 2882 -z ‘zone1’ -c 1 -d ‘/data’ -i ‘em1’ -o __min_full_resource_pool_memory=2147483648,memory_limit=‘8G’,system_memory=‘3G’,datafile_size=‘20G’,log_disk_size=‘15G’,cpu_count=8
[2023-07-26 14:08:33.821] [DEBUG] – exited code 0
[2023-07-26 14:08:33.821] [DEBUG] – root@192.168.111.136 delete env LD_LIBRARY_PATH
[2023-07-26 14:08:33.926] [INFO] observer program health check