OCP搭建好集群后,重启集群OCP服务启动报错 ocp-server-ce-py_script_start-4.2.1 RuntimeError: 'str' object has no attribute 'fetchall'

问题描述
重启集群OCP服务的时候出现报错 ocp-server-ce-py_script_start-4.2.1 RuntimeError: ‘str’ object has no attribute ‘fetchall’
环境为测试环境
由于字数限制,只取了OBD报错相关部分日志,如下:
[2024-03-07 17:12:41.216] [DEBUG] - Deploy status judge
[2024-03-07 17:12:41.217] [INFO] Get local repositories
[2024-03-07 17:12:41.218] [DEBUG] - mkdir /home/admin/.obd/repository
[2024-03-07 17:12:41.219] [DEBUG] - Get local repository oceanbase-ce-4.2.1.2-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.219] [DEBUG] - Search repository oceanbase-ce version: 4.2.1.2, tag: b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4, release: None, package_hash: None
[2024-03-07 17:12:41.219] [DEBUG] - try to get share lock /home/admin/.obd/lock/mirror_and_repo
[2024-03-07 17:12:41.219] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo, count 1
[2024-03-07 17:12:41.219] [DEBUG] - mkdir /home/admin/.obd/repository/oceanbase-ce
[2024-03-07 17:12:41.222] [DEBUG] - Found repository oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.222] [DEBUG] - Get local repository obproxy-ce-4.2.1.0-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.222] [DEBUG] - Search repository obproxy-ce version: 4.2.1.0, tag: 0aed4b782120e4248b749f67be3d2cc82cdcb70d, release: None, package_hash: None
[2024-03-07 17:12:41.222] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo, count 2
[2024-03-07 17:12:41.222] [DEBUG] - mkdir /home/admin/.obd/repository/obproxy-ce
[2024-03-07 17:12:41.227] [DEBUG] - Found repository obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.227] [DEBUG] - Get local repository ocp-server-ce-4.2.1-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.227] [DEBUG] - Search repository ocp-server-ce version: 4.2.1, tag: 58cf72891d75a2fa7c754bafc42d336525baf0b5, release: None, package_hash: None
[2024-03-07 17:12:41.227] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo, count 3
[2024-03-07 17:12:41.227] [DEBUG] - mkdir /home/admin/.obd/repository/ocp-server-ce
[2024-03-07 17:12:41.230] [DEBUG] - Found repository ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.350] [DEBUG] - Get deploy config
[2024-03-07 17:12:41.394] [INFO] Search plugins
[2024-03-07 17:12:41.395] [DEBUG] - Searching start_check plugin for components …
[2024-03-07 17:12:41.396] [DEBUG] - Searching start_check plugin for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.396] [DEBUG] - mkdir /home/admin/.obd/plugins
[2024-03-07 17:12:41.397] [DEBUG] - Found for oceanbase-ce-py_script_start_check-4.2.0.0 for oceanbase-ce-4.2.1.2
[2024-03-07 17:12:41.397] [DEBUG] - Searching start_check plugin for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.398] [DEBUG] - Found for obproxy-ce-py_script_start_check-3.1.0 for obproxy-ce-4.2.1.0
[2024-03-07 17:12:41.398] [DEBUG] - Searching start_check plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.399] [DEBUG] - Found for ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1
[2024-03-07 17:12:41.399] [DEBUG] - Searching create_tenant plugin for components …
[2024-03-07 17:12:41.399] [DEBUG] - Searching create_tenant plugin for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.400] [DEBUG] - Found for oceanbase-ce-py_script_create_tenant-4.2.0.0 for oceanbase-ce-4.2.1.2
[2024-03-07 17:12:41.400] [DEBUG] - Searching create_tenant plugin for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.400] [DEBUG] - No such create_tenant plugin for obproxy-ce-4.2.1.0
[2024-03-07 17:12:41.401] [DEBUG] - Searching create_tenant plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.401] [DEBUG] - No such create_tenant plugin for ocp-server-ce-4.2.1
[2024-03-07 17:12:41.401] [DEBUG] - Searching start plugin for components …
[2024-03-07 17:12:41.401] [DEBUG] - Searching start plugin for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.402] [DEBUG] - Found for oceanbase-ce-py_script_start-4.2.0.0 for oceanbase-ce-4.2.1.2
[2024-03-07 17:12:41.402] [DEBUG] - Searching start plugin for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.403] [DEBUG] - Found for obproxy-ce-py_script_start-3.1.0 for obproxy-ce-4.2.1.0
[2024-03-07 17:12:41.403] [DEBUG] - Searching start plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.403] [DEBUG] - Found for ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1
[2024-03-07 17:12:41.403] [DEBUG] - Searching connect plugin for components …
[2024-03-07 17:12:41.403] [DEBUG] - Searching connect plugin for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.404] [DEBUG] - Found for oceanbase-ce-py_script_connect-3.1.0 for oceanbase-ce-4.2.1.2
[2024-03-07 17:12:41.404] [DEBUG] - Searching connect plugin for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.404] [DEBUG] - Found for obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.2.1.0
[2024-03-07 17:12:41.405] [DEBUG] - Searching connect plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.405] [DEBUG] - Found for ocp-server-ce-py_script_connect-4.2.1 for ocp-server-ce-4.2.1
[2024-03-07 17:12:41.405] [DEBUG] - Searching bootstrap plugin for components …
[2024-03-07 17:12:41.405] [DEBUG] - Searching bootstrap plugin for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.406] [DEBUG] - Found for oceanbase-ce-py_script_bootstrap-4.0.0.0 for oceanbase-ce-4.2.1.2
[2024-03-07 17:12:41.406] [DEBUG] - Searching bootstrap plugin for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.406] [DEBUG] - Found for obproxy-ce-py_script_bootstrap-3.1.0 for obproxy-ce-4.2.1.0
[2024-03-07 17:12:41.406] [DEBUG] - Searching bootstrap plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.407] [DEBUG] - Found for ocp-server-ce-py_script_bootstrap-4.2.1 for ocp-server-ce-4.2.1
[2024-03-07 17:12:41.407] [DEBUG] - Searching display plugin for components …
[2024-03-07 17:12:41.407] [DEBUG] - Searching display plugin for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:41.408] [DEBUG] - Found for oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.2.1.2
[2024-03-07 17:12:41.408] [DEBUG] - Searching display plugin for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:41.408] [DEBUG] - Found for obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.2.1.0
[2024-03-07 17:12:41.408] [DEBUG] - Searching display plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:41.409] [DEBUG] - Found for ocp-server-ce-py_script_display-4.2.1 for ocp-server-ce-4.2.1
[2024-03-07 17:12:41.526] [INFO] Open ssh connection
[2024-03-07 17:12:41.528] [DEBUG] - host: 192.168.142.12, port: 22, user: admin, password: admin
[2024-03-07 17:12:41.659] [INFO] Load cluster param plugin
[2024-03-07 17:12:41.660] [DEBUG] - Searching param plugin for components …
[2024-03-07 17:12:41.660] [DEBUG] - Search param plugin for oceanbase-ce
[2024-03-07 17:12:41.661] [DEBUG] - Found for oceanbase-ce-param-4.2.0.0 for oceanbase-ce-4.2.1.2
[2024-03-07 17:12:41.661] [DEBUG] - Applying oceanbase-ce-param-4.2.0.0 for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:42.599] [DEBUG] - Search param plugin for obproxy-ce
[2024-03-07 17:12:42.599] [DEBUG] - Found for obproxy-ce-param-3.1.0 for obproxy-ce-4.2.1.0
[2024-03-07 17:12:42.599] [DEBUG] - Applying obproxy-ce-param-3.1.0 for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:42.824] [DEBUG] - Search param plugin for ocp-server-ce
[2024-03-07 17:12:42.825] [DEBUG] - Found for ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1
[2024-03-07 17:12:42.825] [DEBUG] - Applying ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:43.024] [DEBUG] - Call oceanbase-ce-py_script_start_check-4.2.0.0 for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:43.024] [DEBUG] - import start_check
[2024-03-07 17:12:43.030] [DEBUG] - add start_check ref count to 1
[2024-03-07 17:12:43.030] [INFO] Check before start observer
[2024-03-07 17:12:43.032] [DEBUG] – admin@192.168.142.12 execute: ls /data/1/clog/tenant_1/
[2024-03-07 17:12:43.060] [DEBUG] – exited code 0
[2024-03-07 17:12:43.060] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/oceanbase/run/observer.pid
[2024-03-07 17:12:43.123] [DEBUG] – exited code 0
[2024-03-07 17:12:43.124] [DEBUG] – admin@192.168.142.12 execute: ls /proc/1736
[2024-03-07 17:12:43.191] [DEBUG] – exited code 0
[2024-03-07 17:12:43.192] [DEBUG] – 192.168.142.12 is runnning, skip
[2024-03-07 17:12:43.192] [DEBUG] – admin@192.168.142.12 execute: cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr
[2024-03-07 17:12:43.258] [DEBUG] – exited code 0
[2024-03-07 17:12:43.258] [DEBUG] – admin@192.168.142.12 execute: ulimit -a
[2024-03-07 17:12:43.320] [DEBUG] – exited code 0
[2024-03-07 17:12:43.320] [DEBUG] – admin@192.168.142.12 execute: cat /proc/meminfo
[2024-03-07 17:12:43.380] [DEBUG] – exited code 0
[2024-03-07 17:12:43.381] [DEBUG] – admin@192.168.142.12 execute: df --block-size=1024
[2024-03-07 17:12:43.447] [DEBUG] – exited code 0
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /, total: 16091447296 avail: 4135329792
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /dev, total: 14648672256 avail: 14648672256
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /dev/shm, total: 14661193728 avail: 14661193728
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /run, total: 14661193728 avail: 14648877056
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /sys/fs/cgroup, total: 14661193728 avail: 14661193728
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /data, total: 214639316992 avail: 160917463040
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /boot, total: 1063256064 avail: 914366464
[2024-03-07 17:12:43.449] [DEBUG] – get disk info for path /run/user/0, total: 2932240384 avail: 2932240384
[2024-03-07 17:12:43.450] [DEBUG] – get disk info for path /run/user/1000, total: 2932240384 avail: 2932240384
[2024-03-07 17:12:43.450] [DEBUG] – disk: {’/’: {‘total’: 16091447296, ‘avail’: 4135329792, ‘need’: 0}, ‘/dev’: {‘total’: 14648672256, ‘avail’: 14648672256, ‘need’: 0}, ‘/dev/shm’: {‘total’: 14661193728, ‘avail’: 14661193728, ‘need’: 0}, ‘/run’: {‘total’: 14661193728, ‘avail’: 14648877056, ‘need’: 0}, ‘/sys/fs/cgroup’: {‘total’: 14661193728, ‘avail’: 14661193728, ‘need’: 0}, ‘/data’: {‘total’: 214639316992, ‘avail’: 160917463040, ‘need’: 0}, ‘/boot’: {‘total’: 1063256064, ‘avail’: 914366464, ‘need’: 0}, ‘/run/user/0’: {‘total’: 2932240384, ‘avail’: 2932240384, ‘need’: 0}, ‘/run/user/1000’: {‘total’: 2932240384, ‘avail’: 2932240384, ‘need’: 0}}
[2024-03-07 17:12:43.450] [DEBUG] – admin@192.168.142.12 execute: date +%s%N
[2024-03-07 17:12:43.513] [DEBUG] – exited code 0
[2024-03-07 17:12:43.513] [DEBUG] – 192.168.142.12 time delta -1.306640625
[2024-03-07 17:12:43.554] [DEBUG] - sub start_check ref count to 0
[2024-03-07 17:12:43.554] [DEBUG] - export start_check
[2024-03-07 17:12:43.555] [DEBUG] - Call obproxy-ce-py_script_start_check-3.1.0 for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:43.555] [DEBUG] - import start_check
[2024-03-07 17:12:43.557] [DEBUG] - add start_check ref count to 1
[2024-03-07 17:12:43.558] [INFO] Check before start obproxy
[2024-03-07 17:12:43.559] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:43.583] [DEBUG] – exited code 0
[2024-03-07 17:12:43.584] [DEBUG] – admin@192.168.142.12 execute: ls /proc/10449/fd
[2024-03-07 17:12:43.650] [DEBUG] – exited code 2, error output:
[2024-03-07 17:12:43.650] [DEBUG] ls: cannot access /proc/10449/fd: No such file or directory
[2024-03-07 17:12:43.650] [DEBUG]
[2024-03-07 17:12:43.650] [DEBUG] – 192.168.142.12 port check
[2024-03-07 17:12:43.650] [DEBUG] – admin@192.168.142.12 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B43’ | awk -F’ ’ ‘{print $2}’ | uniq
[2024-03-07 17:12:43.721] [DEBUG] – exited code 0
[2024-03-07 17:12:43.721] [DEBUG] – admin@192.168.142.12 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B44’ | awk -F’ ’ ‘{print $2}’ | uniq
[2024-03-07 17:12:43.789] [DEBUG] – exited code 0
[2024-03-07 17:12:43.820] [DEBUG] - sub start_check ref count to 0
[2024-03-07 17:12:43.820] [DEBUG] - export start_check
[2024-03-07 17:12:43.820] [DEBUG] - Call ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:43.820] [DEBUG] - import start_check
[2024-03-07 17:12:43.831] [DEBUG] - add start_check ref count to 1
[2024-03-07 17:12:43.832] [INFO] Check before start ocp-server
[2024-03-07 17:12:43.835] [DEBUG] – oceanbase version check
[2024-03-07 17:12:43.835] [DEBUG] – admin@192.168.142.12 execute: sudo -n true
[2024-03-07 17:12:43.875] [DEBUG] – exited code 0
[2024-03-07 17:12:43.876] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/ocp/run/ocp-server.pid
[2024-03-07 17:12:43.945] [DEBUG] – exited code 0
[2024-03-07 17:12:43.945] [DEBUG] – admin@192.168.142.12 execute: ls /proc/8047
[2024-03-07 17:12:44.013] [DEBUG] – exited code 2, error output:
[2024-03-07 17:12:44.013] [DEBUG] ls: cannot access /proc/8047: No such file or directory
[2024-03-07 17:12:44.013] [DEBUG]
[2024-03-07 17:12:44.013] [DEBUG] – admin@192.168.142.12 execute: bash -c ‘cat /proc/net/{udp*,tcp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:1F90’ | awk -F’ ’ ‘{print $2}’ | uniq
[2024-03-07 17:12:44.079] [DEBUG] – exited code 0
[2024-03-07 17:12:44.079] [DEBUG] – admin@192.168.142.12 execute: java -version
[2024-03-07 17:12:44.314] [DEBUG] – exited code 0
[2024-03-07 17:12:44.314] [DEBUG] – java version <ssh.SshReturn object at 0x2ae4753af490>
[2024-03-07 17:12:44.314] [DEBUG] – java_major_version 1.8.0
[2024-03-07 17:12:44.314] [DEBUG] – java_update_version 161
[2024-03-07 17:12:44.314] [DEBUG] – admin@192.168.142.12 execute: which clockdiff
[2024-03-07 17:12:44.380] [DEBUG] – exited code 0
[2024-03-07 17:12:44.381] [DEBUG] – admin@192.168.142.12 execute: cat /proc/meminfo
[2024-03-07 17:12:44.443] [DEBUG] – exited code 0
[2024-03-07 17:12:44.443] [DEBUG] – admin@192.168.142.12 execute: df --block-size=1024
[2024-03-07 17:12:44.508] [DEBUG] – exited code 0
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /, total: 16091447296 avail: 4135329792
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /dev, total: 14648672256 avail: 14648672256
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /dev/shm, total: 14661193728 avail: 14661193728
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /run, total: 14661193728 avail: 14648877056
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /sys/fs/cgroup, total: 14661193728 avail: 14661193728
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /data, total: 214639316992 avail: 160917463040
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /boot, total: 1063256064 avail: 914366464
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /run/user/0, total: 2932240384 avail: 2932240384
[2024-03-07 17:12:44.509] [DEBUG] – get disk info for path /run/user/1000, total: 2932240384 avail: 2932240384
[2024-03-07 17:12:44.509] [DEBUG] – admin@192.168.142.12 execute: df --block-size=1024 /home/admin/logs
[2024-03-07 17:12:44.575] [DEBUG] – exited code 0
[2024-03-07 17:12:44.576] [DEBUG] – get disk info for path /, total: 16091447296 avail: 4135329792
[2024-03-07 17:12:44.615] [DEBUG] - sub start_check ref count to 0
[2024-03-07 17:12:44.615] [DEBUG] - export start_check
[2024-03-07 17:12:44.615] [DEBUG] - Call oceanbase-ce-py_script_start-4.2.0.0 for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:44.615] [DEBUG] - import start
[2024-03-07 17:12:44.619] [DEBUG] - add start ref count to 1
[2024-03-07 17:12:44.620] [INFO] Start observer
[2024-03-07 17:12:44.621] [DEBUG] – admin@192.168.142.12 execute: ls /data/1/clog/tenant_1/
[2024-03-07 17:12:44.645] [DEBUG] – exited code 0
[2024-03-07 17:12:44.646] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/oceanbase/run/observer.pid
[2024-03-07 17:12:44.711] [DEBUG] – exited code 0
[2024-03-07 17:12:44.711] [DEBUG] – admin@192.168.142.12 execute: ls /proc/1736
[2024-03-07 17:12:44.778] [DEBUG] – exited code 0
[2024-03-07 17:12:44.882] [INFO] observer program health check
[2024-03-07 17:12:47.886] [DEBUG] – 192.168.142.12 program health check
[2024-03-07 17:12:47.886] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/oceanbase/run/observer.pid
[2024-03-07 17:12:47.913] [DEBUG] – exited code 0
[2024-03-07 17:12:47.913] [DEBUG] – admin@192.168.142.12 execute: ls /proc/1736
[2024-03-07 17:12:47.976] [DEBUG] – exited code 0
[2024-03-07 17:12:47.976] [DEBUG] – 192.168.142.12 observer[pid: 1736] started
[2024-03-07 17:12:48.016] [DEBUG] - sub start ref count to 0
[2024-03-07 17:12:48.016] [DEBUG] - export start
[2024-03-07 17:12:48.016] [DEBUG] - Call oceanbase-ce-py_script_connect-3.1.0 for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:12:48.016] [DEBUG] - import connect
[2024-03-07 17:12:48.019] [DEBUG] - add connect ref count to 1
[2024-03-07 17:12:48.021] [INFO] Connect to observer
[2024-03-07 17:12:48.022] [DEBUG] – connect 192.168.142.12 -P2881 -uroot -pzzh!@#123QAZ
[2024-03-07 17:12:48.028] [DEBUG] – execute sql: select 1. args: None
[2024-03-07 17:12:48.152] [DEBUG] - sub connect ref count to 0
[2024-03-07 17:12:48.152] [DEBUG] - export connect
[2024-03-07 17:12:48.152] [DEBUG] - Call obproxy-ce-py_script_start-3.1.0 for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:48.153] [DEBUG] - import start
[2024-03-07 17:12:48.155] [DEBUG] - add start ref count to 1
[2024-03-07 17:12:48.156] [INFO] Start obproxy
[2024-03-07 17:12:48.156] [DEBUG] – admin@192.168.142.12 execute: ls /home/admin/obproxy/etc/obproxy_config.bin
[2024-03-07 17:12:48.177] [DEBUG] – exited code 0
[2024-03-07 17:12:48.177] [DEBUG] – 192.168.142.12 port check
[2024-03-07 17:12:48.177] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:48.245] [DEBUG] – exited code 0
[2024-03-07 17:12:48.246] [DEBUG] – admin@192.168.142.12 execute: ls /proc/10449/
[2024-03-07 17:12:48.312] [DEBUG] – exited code 2, error output:
[2024-03-07 17:12:48.312] [DEBUG] ls: cannot access /proc/10449/: No such file or directory
[2024-03-07 17:12:48.312] [DEBUG]
[2024-03-07 17:12:48.312] [DEBUG] – starting 192.168.142.12 obproxy
[2024-03-07 17:12:48.313] [DEBUG] – admin@192.168.142.12 set env LD_LIBRARY_PATH to ‘/home/admin/obproxy/lib:’
[2024-03-07 17:12:48.313] [DEBUG] – admin@192.168.142.12 execute: cd /home/admin/obproxy; /home/admin/obproxy/bin/obproxy -o obproxy_sys_password=‘714395d584f836b68c2d31651e30bb74e3bb4a20’,enable_strict_kernel_release=False,skip_proxy_sys_private_check=True,enable_cluster_checkout=False --listen_port 2883 --prometheus_listen_port 2884 --rs_list ‘192.168.142.12:2881’ --cluster_name ‘obcluster’
[2024-03-07 17:12:48.463] [DEBUG] – exited code 0
[2024-03-07 17:12:48.463] [DEBUG] – admin@192.168.142.12 delete env LD_LIBRARY_PATH
[2024-03-07 17:12:48.463] [DEBUG] – admin@192.168.142.12 execute: ps -aux | grep -e ‘/home/admin/obproxy/bin/obproxy -o obproxy_sys_password=714395d584f836b68c2d31651e30bb74e3bb4a20,enable_strict_kernel_release=False,skip_proxy_sys_private_check=True,enable_cluster_checkout=False --listen_port 2883 --prometheus_listen_port 2884 --rs_list 192.168.142.12:2881 --cluster_name obcluster$’ | grep -v grep | awk ‘{print $2}’ > /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:48.562] [DEBUG] – exited code 0
[2024-03-07 17:12:48.679] [INFO] obproxy program health check
[2024-03-07 17:12:48.680] [DEBUG] – 192.168.142.12 program health check
[2024-03-07 17:12:48.680] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:48.707] [DEBUG] – exited code 0
[2024-03-07 17:12:48.708] [DEBUG] – admin@192.168.142.12 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B43’ | awk -F’ ’ ‘{print $2}’ | uniq
[2024-03-07 17:12:48.779] [DEBUG] – exited code 0
[2024-03-07 17:12:48.779] [DEBUG] – failed to start 192.168.142.12 obproxy, remaining retries: 299
[2024-03-07 17:12:49.780] [DEBUG] – 192.168.142.12 program health check
[2024-03-07 17:12:49.780] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:49.805] [DEBUG] – exited code 0
[2024-03-07 17:12:49.806] [DEBUG] – admin@192.168.142.12 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B43’ | awk -F’ ’ ‘{print $2}’ | uniq
[2024-03-07 17:12:49.873] [DEBUG] – exited code 0
[2024-03-07 17:12:49.873] [DEBUG] – 43009
[2024-03-07 17:12:49.873] [DEBUG]
[2024-03-07 17:12:49.873] [DEBUG] – admin@192.168.142.12 execute: ls -l /proc/4410/fd/ |grep -E ‘socket:[(43009)]’
[2024-03-07 17:12:49.940] [DEBUG] – exited code 0
[2024-03-07 17:12:49.940] [DEBUG] – admin@192.168.142.12 execute: pid=cat /home/admin/obproxy/run/obproxyd-192.168.142.12-2883.pid && ls /proc/$pid
[2024-03-07 17:12:50.007] [DEBUG] – exited code 2, error output:
[2024-03-07 17:12:50.008] [DEBUG] ls: cannot access /proc/10428: No such file or directory
[2024-03-07 17:12:50.008] [DEBUG]
[2024-03-07 17:12:50.008] [DEBUG] – admin@192.168.142.12 execute: echo 4410 > /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:50.071] [DEBUG] – exited code 0
[2024-03-07 17:12:50.162] [DEBUG] - current remote_transporter RemoteTransporter.CLIENT
[2024-03-07 17:12:50.163] [DEBUG] – admin@192.168.142.12 execute: mkdir -p /home/admin/obproxy && rm -fr /home/admin/obproxy/obproxyd.sh
[2024-03-07 17:12:50.190] [DEBUG] – exited code 0
[2024-03-07 17:12:50.190] [DEBUG] – send /home/admin/.obd/plugins/obproxy-ce/3.1.0/obproxyd.sh to /home/admin/obproxy/obproxyd.sh
[2024-03-07 17:12:50.234] [DEBUG] - admin@192.168.142.12 execute: chmod 644 /home/admin/obproxy/obproxyd.sh
[2024-03-07 17:12:50.258] [DEBUG] - exited code 0
[2024-03-07 17:12:50.259] [DEBUG] – admin@192.168.142.12 execute: bash /home/admin/obproxy/obproxyd.sh /home/admin/obproxy 192.168.142.12 2883
[2024-03-07 17:12:50.327] [DEBUG] – exited code 0
[2024-03-07 17:12:51.329] [DEBUG] – 192.168.142.12 program health check
[2024-03-07 17:12:51.329] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:51.355] [DEBUG] – exited code 0
[2024-03-07 17:12:51.356] [DEBUG] – admin@192.168.142.12 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B43’ | awk -F’ ’ ‘{print $2}’ | uniq
[2024-03-07 17:12:51.425] [DEBUG] – exited code 0
[2024-03-07 17:12:51.425] [DEBUG] – failed to start 192.168.142.12 obproxy, remaining retries: 297
[2024-03-07 17:12:52.427] [DEBUG] – 192.168.142.12 program health check
[2024-03-07 17:12:52.427] [DEBUG] – admin@192.168.142.12 execute: cat /home/admin/obproxy/run/obproxy-192.168.142.12-2883.pid
[2024-03-07 17:12:52.453] [DEBUG] – exited code 0
[2024-03-07 17:12:52.454] [DEBUG] – admin@192.168.142.12 execute: bash -c ‘cat /proc/net/{tcp*,udp*}’ | awk -F’ ’ ‘{print $2,$10}’ | grep ‘00000000:0B43’ | awk -F’ ’ ‘{print $2}’ | uniq
[2024-03-07 17:12:52.523] [DEBUG] – exited code 0
[2024-03-07 17:12:52.523] [DEBUG] – 43033
[2024-03-07 17:12:52.523] [DEBUG]
[2024-03-07 17:12:52.523] [DEBUG] – admin@192.168.142.12 execute: ls -l /proc/4635/fd/ |grep -E ‘socket:[(43033)]’
[2024-03-07 17:12:52.592] [DEBUG] – exited code 0
[2024-03-07 17:12:52.593] [DEBUG] – admin@192.168.142.12 execute: pid=cat /home/admin/obproxy/run/obproxyd-192.168.142.12-2883.pid && ls /proc/$pid
[2024-03-07 17:12:52.661] [DEBUG] – exited code 0
[2024-03-07 17:12:52.661] [DEBUG] – 192.168.142.12 obproxy[pid: 4635] started
[2024-03-07 17:12:52.724] [DEBUG] - sub start ref count to 0
[2024-03-07 17:12:52.724] [DEBUG] - export start
[2024-03-07 17:12:52.725] [DEBUG] - Call obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:52.725] [DEBUG] - import connect
[2024-03-07 17:12:52.728] [DEBUG] - add connect ref count to 1
[2024-03-07 17:12:52.729] [INFO] Connect to obproxy
[2024-03-07 17:12:52.731] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@proxysys -pwVlp3y6wkY
[2024-03-07 17:12:52.861] [DEBUG] - sub connect ref count to 0
[2024-03-07 17:12:52.861] [DEBUG] - export connect
[2024-03-07 17:12:52.862] [INFO] Initialize obproxy-ce
[2024-03-07 17:12:52.863] [DEBUG] - Call obproxy-ce-py_script_bootstrap-3.1.0 for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:12:52.863] [DEBUG] - import bootstrap
[2024-03-07 17:12:52.865] [DEBUG] - add bootstrap ref count to 1
[2024-03-07 17:12:52.865] [DEBUG] – execute sql: alter proxyconfig set observer_sys_password = %s. args: [‘eRQ8xFVF9R’]
[2024-03-07 17:12:52.870] [DEBUG] - sub bootstrap ref count to 0
[2024-03-07 17:12:52.871] [DEBUG] - export bootstrap
[2024-03-07 17:12:52.995] [DEBUG] - Call ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5
[2024-03-07 17:12:52.995] [DEBUG] - import start
[2024-03-07 17:12:53.008] [DEBUG] - add start ref count to 1
[2024-03-07 17:12:53.009] [DEBUG] – metadb connect check
[2024-03-07 17:12:55.010] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:12:57.153] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:00.159] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:03.166] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:06.166] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:09.168] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:12.167] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:15.169] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:18.169] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:21.170] [DEBUG] – connect 192.168.142.12 -P2883 -uroot@sys -pzzh!@#123QAZ
[2024-03-07 17:13:22.166] [ERROR] ocp-server-ce-py_script_start-4.2.1 RuntimeError: ‘str’ object has no attribute ‘fetchall’
[2024-03-07 17:13:22.166] [ERROR] Traceback (most recent call last):
[2024-03-07 17:13:22.166] [ERROR] File “core.py”, line 1673, in start_cluster
[2024-03-07 17:13:22.166] [ERROR] File “core.py”, line 1758, in _start_cluster
[2024-03-07 17:13:22.166] [ERROR] File “core.py”, line 184, in call_plugin
[2024-03-07 17:13:22.166] [ERROR] File “_plugin.py”, line 324, in call
[2024-03-07 17:13:22.166] [ERROR] File “_plugin.py”, line 286, in _new_func
[2024-03-07 17:13:22.166] [ERROR] File “/home/admin/.obd/plugins/ocp-server-ce/4.2.1/start.py”, line 704, in start
[2024-03-07 17:13:22.166] [ERROR] if not start_cluster():
[2024-03-07 17:13:22.166] [ERROR] File “/home/admin/.obd/plugins/ocp-server-ce/4.2.1/start.py”, line 503, in start_cluster
[2024-03-07 17:13:22.166] [ERROR] if not create_tenant(cursor, meta_tenant, meta_max_cpu, parse_size(meta_memory_size), meta_user,
[2024-03-07 17:13:22.166] [ERROR] File “/home/admin/.obd/plugins/ocp-server-ce/4.2.1/start.py”, line 239, in create_tenant
[2024-03-07 17:13:22.166] [ERROR] res = cursor.fetchall(sql)
[2024-03-07 17:13:22.166] [ERROR] AttributeError: ‘str’ object has no attribute ‘fetchall’
[2024-03-07 17:13:22.166] [ERROR]
[2024-03-07 17:13:22.166] [DEBUG] - sub start ref count to 0
[2024-03-07 17:13:22.166] [DEBUG] - export start
[2024-03-07 17:13:22.166] [ERROR] ocp-server-ce start failed
[2024-03-07 17:13:22.166] [DEBUG] - Call oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.2.1.2-102000042023120514.el7-b2ccb524f200a9ef0fad2cddf59d309ddaa2e3e4
[2024-03-07 17:13:22.166] [DEBUG] - import display
[2024-03-07 17:13:22.168] [DEBUG] - add display ref count to 1
[2024-03-07 17:13:22.168] [INFO] Wait for observer init
[2024-03-07 17:13:22.169] [DEBUG] – execute sql: select * from oceanbase.__all_server. args: None
[2024-03-07 17:13:22.174] [INFO] ±-------------------------------------------------+
[2024-03-07 17:13:22.174] [INFO] | observer |
[2024-03-07 17:13:22.174] [INFO] ±---------------±--------±-----±------±-------+
[2024-03-07 17:13:22.174] [INFO] | ip | version | port | zone | status |
[2024-03-07 17:13:22.174] [INFO] ±---------------±--------±-----±------±-------+
[2024-03-07 17:13:22.174] [INFO] | 192.168.142.12 | 4.2.1.2 | 2881 | zone1 | ACTIVE |
[2024-03-07 17:13:22.174] [INFO] ±---------------±--------±-----±------±-------+
[2024-03-07 17:13:22.174] [INFO] obclient -h192.168.142.12 -P2881 -uroot -p’zzh!@#123QAZ’ -Doceanbase -A
[2024-03-07 17:13:22.300] [INFO] ±-------------------------------------------------+
[2024-03-07 17:13:22.300] [INFO] | observer |
[2024-03-07 17:13:22.300] [INFO] ±---------------±--------±-----±------±-------+
[2024-03-07 17:13:22.301] [INFO] | ip | version | port | zone | status |
[2024-03-07 17:13:22.301] [INFO] ±---------------±--------±-----±------±-------+
[2024-03-07 17:13:22.301] [INFO] | 192.168.142.12 | 4.2.1.2 | 2881 | zone1 | ACTIVE |
[2024-03-07 17:13:22.301] [INFO] ±---------------±--------±-----±------±-------+
[2024-03-07 17:13:22.301] [INFO] obclient -h192.168.142.12 -P2881 -uroot -p’zzh!@#123QAZ’ -Doceanbase -A
[2024-03-07 17:13:22.301] [INFO]
[2024-03-07 17:13:22.301] [DEBUG] - sub display ref count to 0
[2024-03-07 17:13:22.301] [DEBUG] - export display
[2024-03-07 17:13:22.302] [DEBUG] - Call obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.2.1.0-11.el7-0aed4b782120e4248b749f67be3d2cc82cdcb70d
[2024-03-07 17:13:22.302] [DEBUG] - import display
[2024-03-07 17:13:22.304] [DEBUG] - add display ref count to 1
[2024-03-07 17:13:22.305] [DEBUG] – execute sql: show proxyconfig like “%port”. args: None
[2024-03-07 17:13:22.318] [INFO] ±-------------------------------------------------+
[2024-03-07 17:13:22.318] [INFO] | obproxy |
[2024-03-07 17:13:22.318] [INFO] ±---------------±-----±----------------±-------+
[2024-03-07 17:13:22.318] [INFO] | ip | port | prometheus_port | status |
[2024-03-07 17:13:22.318] [INFO] ±---------------±-----±----------------±-------+
[2024-03-07 17:13:22.318] [INFO] | 192.168.142.12 | 2883 | 2884 | active |
[2024-03-07 17:13:22.318] [INFO] ±---------------±-----±----------------±-------+
[2024-03-07 17:13:22.319] [INFO] obclient -h192.168.142.12 -P2883 -uroot -p’zzh!@#123QAZ’ -Doceanbase -A
[2024-03-07 17:13:22.319] [DEBUG] - sub display ref count to 0
[2024-03-07 17:13:22.319] [DEBUG] - export display
[2024-03-07 17:13:22.321] [INFO] See OceanBase分布式数据库-海量数据 笔笔算数 .
[2024-03-07 17:13:22.321] [INFO] Trace ID: d9adbdfe-dc62-11ee-a789-000c29848dcc
[2024-03-07 17:13:22.321] [INFO] If you want to view detailed obd logs, please run: obd display-trace d9adbdfe-dc62-11ee-a789-000c29848dcc
[2024-03-07 17:13:22.323] [DEBUG] - unlock /home/admin/.obd/lock/global
[2024-03-07 17:13:22.323] [DEBUG] - unlock /home/admin/.obd/lock/deploy_obcluster
[2024-03-07 17:13:22.323] [DEBUG] - unlock /home/admin/.obd/lock/mirror_and_repo

请教下各位,遇到该问题如何处理解决。

1 个赞

收到,正在联系相关同学确认,请稍等。

2 个赞

rm -f ~/.obd/version 文件
obd mirror list 后
重新启动看下。

1 个赞

可以解决