[2024-01-25 09:59:28.229] [DEBUG] - mkdir /home/admin/.obd/lock/ [2024-01-25 09:59:28.229] [DEBUG] - unknown lock mode [2024-01-25 09:59:28.229] [DEBUG] - try to get share lock /home/admin/.obd/lock/global [2024-01-25 09:59:28.229] [DEBUG] - share lock `/home/admin/.obd/lock/global`, count 1 [2024-01-25 09:59:28.229] [DEBUG] - cmd: ['obtest'] [2024-01-25 09:59:28.229] [DEBUG] - opts: {'servers': None, 'components': None, 'force_delete': None, 'strict_check': None, 'without_parameter': True} [2024-01-25 09:59:28.229] [DEBUG] - Get Deploy by name [2024-01-25 09:59:28.229] [DEBUG] - mkdir /home/admin/.obd/cluster/ [2024-01-25 09:59:28.230] [DEBUG] - mkdir /home/admin/.obd/config_parser/ [2024-01-25 09:59:28.230] [DEBUG] - try to get exclusive lock /home/admin/.obd/lock/deploy_obtest [2024-01-25 09:59:28.230] [DEBUG] - exclusive lock `/home/admin/.obd/lock/deploy_obtest`, count 1 [2024-01-25 09:59:28.234] [DEBUG] - Deploy status judge [2024-01-25 09:59:28.234] [INFO] Get local repositories [2024-01-25 09:59:28.235] [DEBUG] - mkdir /home/admin/.obd/repository [2024-01-25 09:59:28.235] [DEBUG] - Get local repository oceanbase-ce-4.1.0.0-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.235] [DEBUG] - Search repository oceanbase-ce version: 4.1.0.0, tag: d598937b1cfb1df85e2c2231acf024e4994db533, release: None, package_hash: None [2024-01-25 09:59:28.236] [DEBUG] - try to get share lock /home/admin/.obd/lock/mirror_and_repo [2024-01-25 09:59:28.236] [DEBUG] - share lock `/home/admin/.obd/lock/mirror_and_repo`, count 1 [2024-01-25 09:59:28.236] [DEBUG] - mkdir /home/admin/.obd/repository/oceanbase-ce [2024-01-25 09:59:28.238] [DEBUG] - Found repository oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.238] [DEBUG] - Get local repository obproxy-ce-4.1.0.0-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.238] [DEBUG] - Search repository obproxy-ce version: 4.1.0.0, tag: 2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe, release: None, package_hash: None [2024-01-25 09:59:28.238] [DEBUG] - share lock `/home/admin/.obd/lock/mirror_and_repo`, count 2 [2024-01-25 09:59:28.238] [DEBUG] - mkdir /home/admin/.obd/repository/obproxy-ce [2024-01-25 09:59:28.239] [DEBUG] - Found repository obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.366] [DEBUG] - Get deploy config [2024-01-25 09:59:28.432] [INFO] Search plugins [2024-01-25 09:59:28.433] [DEBUG] - Searching start_check plugin for components ... [2024-01-25 09:59:28.433] [DEBUG] - Searching start_check plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.433] [DEBUG] - mkdir /home/admin/.obd/plugins [2024-01-25 09:59:28.435] [DEBUG] - Found for oceanbase-ce-py_script_start_check-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-01-25 09:59:28.435] [DEBUG] - Searching start_check plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.435] [DEBUG] - Found for obproxy-ce-py_script_start_check-3.1.0 for obproxy-ce-4.1.0.0 [2024-01-25 09:59:28.435] [DEBUG] - Searching create_tenant plugin for components ... [2024-01-25 09:59:28.435] [DEBUG] - Searching create_tenant plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.436] [DEBUG] - Found for oceanbase-ce-py_script_create_tenant-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-01-25 09:59:28.436] [DEBUG] - Searching create_tenant plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.436] [DEBUG] - No such create_tenant plugin for obproxy-ce-4.1.0.0 [2024-01-25 09:59:28.436] [DEBUG] - Searching start plugin for components ... [2024-01-25 09:59:28.436] [DEBUG] - Searching start plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.437] [DEBUG] - Found for oceanbase-ce-py_script_start-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-01-25 09:59:28.437] [DEBUG] - Searching start plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.437] [DEBUG] - Found for obproxy-ce-py_script_start-3.1.0 for obproxy-ce-4.1.0.0 [2024-01-25 09:59:28.437] [DEBUG] - Searching connect plugin for components ... [2024-01-25 09:59:28.437] [DEBUG] - Searching connect plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.438] [DEBUG] - Found for oceanbase-ce-py_script_connect-3.1.0 for oceanbase-ce-4.1.0.0 [2024-01-25 09:59:28.438] [DEBUG] - Searching connect plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.438] [DEBUG] - Found for obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.1.0.0 [2024-01-25 09:59:28.439] [DEBUG] - Searching bootstrap plugin for components ... [2024-01-25 09:59:28.439] [DEBUG] - Searching bootstrap plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.439] [DEBUG] - Found for oceanbase-ce-py_script_bootstrap-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-01-25 09:59:28.439] [DEBUG] - Searching bootstrap plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.440] [DEBUG] - Found for obproxy-ce-py_script_bootstrap-3.1.0 for obproxy-ce-4.1.0.0 [2024-01-25 09:59:28.440] [DEBUG] - Searching display plugin for components ... [2024-01-25 09:59:28.440] [DEBUG] - Searching display plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:28.441] [DEBUG] - Found for oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.1.0.0 [2024-01-25 09:59:28.441] [DEBUG] - Searching display plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:28.441] [DEBUG] - Found for obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.1.0.0 [2024-01-25 09:59:28.565] [INFO] Open ssh connection [2024-01-25 09:59:29.089] [INFO] Load cluster param plugin [2024-01-25 09:59:29.089] [DEBUG] - Searching param plugin for components ... [2024-01-25 09:59:29.090] [DEBUG] - Search param plugin for oceanbase-ce [2024-01-25 09:59:29.090] [DEBUG] - Found for oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-01-25 09:59:29.091] [DEBUG] - Applying oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:29.602] [DEBUG] - Search param plugin for obproxy-ce [2024-01-25 09:59:29.602] [DEBUG] - Found for obproxy-ce-param-3.1.0 for obproxy-ce-4.1.0.0 [2024-01-25 09:59:29.602] [DEBUG] - Applying obproxy-ce-param-3.1.0 for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:29.843] [DEBUG] - Call oceanbase-ce-py_script_start_check-4.0.0.0 for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:29.843] [DEBUG] - import start_check [2024-01-25 09:59:29.854] [DEBUG] - add start_check ref count to 1 [2024-01-25 09:59:29.854] [INFO] Check before start observer [2024-01-25 09:59:29.856] [DEBUG] -- admin@192.168.10.168 execute: ls /data/oceanbase//clog/tenant_1/ [2024-01-25 09:59:29.882] [DEBUG] -- exited code 0 [2024-01-25 09:59:29.883] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:29.946] [DEBUG] -- exited code 0 [2024-01-25 09:59:29.947] [DEBUG] -- admin@192.168.10.168 execute: ls /proc/40237 [2024-01-25 09:59:30.008] [DEBUG] -- exited code 2, error output: [2024-01-25 09:59:30.008] [DEBUG] ls: cannot access /proc/40237: No such file or directory [2024-01-25 09:59:30.009] [DEBUG] [2024-01-25 09:59:30.009] [DEBUG] -- 192.168.10.168(192.168.10.168) port check [2024-01-25 09:59:30.009] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3251' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:30.079] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.080] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3252' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:30.148] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.149] [DEBUG] -- admin@192.168.10.168 execute: ls /data/oceanbase//sstable/block_file [2024-01-25 09:59:30.214] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.215] [DEBUG] -- admin@192.168.10.169 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:30.288] [DEBUG] -- exited code 1, error output: [2024-01-25 09:59:30.289] [DEBUG] cat: /home/admin/oceanbase-ce/run/observer.pid: No such file or directory [2024-01-25 09:59:30.289] [DEBUG] [2024-01-25 09:59:30.289] [DEBUG] -- 192.168.10.169(192.168.10.169) port check [2024-01-25 09:59:30.290] [DEBUG] -- admin@192.168.10.169 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3251' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:30.416] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.417] [DEBUG] -- admin@192.168.10.169 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3252' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:30.537] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.538] [DEBUG] -- admin@192.168.10.169 execute: ls /data/oceanbase//sstable/block_file [2024-01-25 09:59:30.651] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.653] [DEBUG] -- admin@192.168.10.172 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:30.814] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.814] [DEBUG] -- admin@192.168.10.172 execute: ls /proc/23253 [2024-01-25 09:59:30.976] [DEBUG] -- exited code 0 [2024-01-25 09:59:30.976] [DEBUG] -- 192.168.10.172(192.168.10.172) is runnning, skip [2024-01-25 09:59:30.976] [DEBUG] -- admin@192.168.10.168 execute: cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr [2024-01-25 09:59:31.002] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.003] [DEBUG] -- admin@192.168.10.168 execute: ulimit -a [2024-01-25 09:59:31.066] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.067] [DEBUG] -- admin@192.168.10.168 execute: cat /proc/meminfo [2024-01-25 09:59:31.128] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.129] [DEBUG] -- admin@192.168.10.168 execute: df --block-size=1024 [2024-01-25 09:59:31.190] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.191] [DEBUG] -- get disk info for path /dev, total: 67449462784 avail: 67449462784 [2024-01-25 09:59:31.191] [DEBUG] -- get disk info for path /dev/shm, total: 67461758976 avail: 67461758976 [2024-01-25 09:59:31.191] [DEBUG] -- get disk info for path /run, total: 67461758976 avail: 67442491392 [2024-01-25 09:59:31.191] [DEBUG] -- get disk info for path /sys/fs/cgroup, total: 67461758976 avail: 67461758976 [2024-01-25 09:59:31.192] [DEBUG] -- get disk info for path /, total: 676431134720 avail: 307159482368 [2024-01-25 09:59:31.192] [DEBUG] -- get disk info for path /boot, total: 1063256064 avail: 906252288 [2024-01-25 09:59:31.192] [DEBUG] -- get disk info for path /data, total: 1770675367936 avail: 266268119040 [2024-01-25 09:59:31.192] [DEBUG] -- get disk info for path /data/nfs_server, total: 885273681920 avail: 567370678272 [2024-01-25 09:59:31.192] [DEBUG] -- get disk info for path /home, total: 210350637056 avail: 50637463552 [2024-01-25 09:59:31.192] [DEBUG] -- get disk info for path /run/user/0, total: 13492355072 avail: 13492355072 [2024-01-25 09:59:31.192] [DEBUG] -- get disk info for path /run/user/1003, total: 13492355072 avail: 13492355072 [2024-01-25 09:59:31.192] [DEBUG] -- disk: {'/dev': {'total': 67449462784, 'avail': 67449462784, 'need': 0}, '/dev/shm': {'total': 67461758976, 'avail': 67461758976, 'need': 0}, '/run': {'total': 67461758976, 'avail': 67442491392, 'need': 0}, '/sys/fs/cgroup': {'total': 67461758976, 'avail': 67461758976, 'need': 0}, '/': {'total': 676431134720, 'avail': 307159482368, 'need': 0}, '/boot': {'total': 1063256064, 'avail': 906252288, 'need': 0}, '/data': {'total': 1770675367936, 'avail': 266268119040, 'need': 0}, '/data/nfs_server': {'total': 885273681920, 'avail': 567370678272, 'need': 0}, '/home': {'total': 210350637056, 'avail': 50637463552, 'need': 0}, '/run/user/0': {'total': 13492355072, 'avail': 13492355072, 'need': 0}, '/run/user/1003': {'total': 13492355072, 'avail': 13492355072, 'need': 0}} [2024-01-25 09:59:31.193] [DEBUG] -- admin@192.168.10.169 execute: cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr [2024-01-25 09:59:31.267] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.268] [DEBUG] -- admin@192.168.10.169 execute: ulimit -a [2024-01-25 09:59:31.378] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.379] [WARNING] OBD-1007: (192.168.10.169) The recommended number of open files is 655350 (Current value: 400000) [2024-01-25 09:59:31.379] [DEBUG] -- admin@192.168.10.169 execute: cat /proc/meminfo [2024-01-25 09:59:31.490] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.490] [DEBUG] -- admin@192.168.10.169 execute: df --block-size=1024 [2024-01-25 09:59:31.607] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.607] [DEBUG] -- get disk info for path /dev, total: 67449462784 avail: 67449462784 [2024-01-25 09:59:31.607] [DEBUG] -- get disk info for path /dev/shm, total: 67461763072 avail: 67461677056 [2024-01-25 09:59:31.608] [DEBUG] -- get disk info for path /run, total: 67461763072 avail: 64741265408 [2024-01-25 09:59:31.608] [DEBUG] -- get disk info for path /sys/fs/cgroup, total: 67461763072 avail: 67461763072 [2024-01-25 09:59:31.608] [DEBUG] -- get disk info for path /, total: 697905971200 avail: 585285246976 [2024-01-25 09:59:31.608] [DEBUG] -- get disk info for path /boot, total: 1063256064 avail: 906256384 [2024-01-25 09:59:31.608] [DEBUG] -- get disk info for path /home, total: 196126314496 avail: 137725927424 [2024-01-25 09:59:31.608] [DEBUG] -- get disk info for path /data, total: 2656076316672 avail: 351378292736 [2024-01-25 09:59:31.609] [DEBUG] -- get disk info for path /run/user/1006, total: 13492355072 avail: 13492355072 [2024-01-25 09:59:31.609] [DEBUG] -- get disk info for path /run/user/0, total: 13492355072 avail: 13492355072 [2024-01-25 09:59:31.609] [DEBUG] -- get disk info for path /run/user/1007, total: 13492355072 avail: 13492355072 [2024-01-25 09:59:31.609] [DEBUG] -- get disk info for path /run/user/1001, total: 13492355072 avail: 13492355072 [2024-01-25 09:59:31.609] [DEBUG] -- disk: {'/dev': {'total': 67449462784, 'avail': 67449462784, 'need': 0}, '/dev/shm': {'total': 67461763072, 'avail': 67461677056, 'need': 0}, '/run': {'total': 67461763072, 'avail': 64741265408, 'need': 0}, '/sys/fs/cgroup': {'total': 67461763072, 'avail': 67461763072, 'need': 0}, '/': {'total': 697905971200, 'avail': 585285246976, 'need': 0}, '/boot': {'total': 1063256064, 'avail': 906256384, 'need': 0}, '/home': {'total': 196126314496, 'avail': 137725927424, 'need': 0}, '/data': {'total': 2656076316672, 'avail': 351378292736, 'need': 0}, '/run/user/1006': {'total': 13492355072, 'avail': 13492355072, 'need': 0}, '/run/user/0': {'total': 13492355072, 'avail': 13492355072, 'need': 0}, '/run/user/1007': {'total': 13492355072, 'avail': 13492355072, 'need': 0}, '/run/user/1001': {'total': 13492355072, 'avail': 13492355072, 'need': 0}} [2024-01-25 09:59:31.609] [DEBUG] -- admin@192.168.10.168 execute: date +%s%N [2024-01-25 09:59:31.635] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.636] [DEBUG] -- 192.168.10.168 time delta -1.738037109375 [2024-01-25 09:59:31.636] [DEBUG] -- admin@192.168.10.169 execute: date +%s%N [2024-01-25 09:59:31.718] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.718] [DEBUG] -- 192.168.10.169 time delta -18.487060546875 [2024-01-25 09:59:31.719] [DEBUG] -- admin@192.168.10.172 execute: date +%s%N [2024-01-25 09:59:31.877] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.878] [DEBUG] -- 192.168.10.172 time delta -47.164306640625 [2024-01-25 09:59:31.942] [INFO] [WARN] OBD-1007: (192.168.10.169) The recommended number of open files is 655350 (Current value: 400000) [2024-01-25 09:59:31.942] [INFO] [2024-01-25 09:59:31.942] [DEBUG] - sub start_check ref count to 0 [2024-01-25 09:59:31.942] [DEBUG] - export start_check [2024-01-25 09:59:31.943] [DEBUG] - Call obproxy-ce-py_script_start_check-3.1.0 for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:31.943] [DEBUG] - import start_check [2024-01-25 09:59:31.946] [DEBUG] - add start_check ref count to 1 [2024-01-25 09:59:31.947] [INFO] Check before start obproxy [2024-01-25 09:59:31.948] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:31.973] [DEBUG] -- exited code 0 [2024-01-25 09:59:31.973] [DEBUG] -- admin@192.168.10.168 execute: ls /proc/41413/fd [2024-01-25 09:59:32.037] [DEBUG] -- exited code 2, error output: [2024-01-25 09:59:32.037] [DEBUG] ls: cannot access /proc/41413/fd: No such file or directory [2024-01-25 09:59:32.037] [DEBUG] [2024-01-25 09:59:32.038] [DEBUG] -- 192.168.10.168 port check [2024-01-25 09:59:32.038] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3253' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:32.110] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.110] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3254' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:32.180] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.209] [DEBUG] - sub start_check ref count to 0 [2024-01-25 09:59:32.209] [DEBUG] - export start_check [2024-01-25 09:59:32.210] [DEBUG] - Call oceanbase-ce-py_script_start-4.0.0.0 for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:32.210] [DEBUG] - import start [2024-01-25 09:59:32.213] [DEBUG] - add start ref count to 1 [2024-01-25 09:59:32.214] [INFO] Start observer [2024-01-25 09:59:32.215] [DEBUG] -- admin@192.168.10.168 execute: ls /data/oceanbase//clog/tenant_1/ [2024-01-25 09:59:32.241] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.242] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:32.303] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.304] [DEBUG] -- admin@192.168.10.168 execute: ls /proc/40237 [2024-01-25 09:59:32.369] [DEBUG] -- exited code 2, error output: [2024-01-25 09:59:32.369] [DEBUG] ls: cannot access /proc/40237: No such file or directory [2024-01-25 09:59:32.369] [DEBUG] [2024-01-25 09:59:32.370] [DEBUG] -- 192.168.10.168(192.168.10.168) start command construction [2024-01-25 09:59:32.370] [DEBUG] -- admin@192.168.10.168 execute: ls /home/admin/oceanbase-ce/etc/observer.config.bin [2024-01-25 09:59:32.432] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.433] [DEBUG] -- admin@192.168.10.169 execute: ls /data/oceanbase//clog/tenant_1/ [2024-01-25 09:59:32.508] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.509] [DEBUG] -- admin@192.168.10.169 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:32.619] [DEBUG] -- exited code 1, error output: [2024-01-25 09:59:32.620] [DEBUG] cat: /home/admin/oceanbase-ce/run/observer.pid: No such file or directory [2024-01-25 09:59:32.620] [DEBUG] [2024-01-25 09:59:32.620] [DEBUG] -- 192.168.10.169(192.168.10.169) start command construction [2024-01-25 09:59:32.621] [DEBUG] -- admin@192.168.10.169 execute: ls /home/admin/oceanbase-ce/etc/observer.config.bin [2024-01-25 09:59:32.733] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.734] [DEBUG] -- admin@192.168.10.172 execute: ls /data/oceanbase//clog/tenant_1/ [2024-01-25 09:59:32.892] [DEBUG] -- exited code 0 [2024-01-25 09:59:32.893] [DEBUG] -- admin@192.168.10.172 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:33.065] [DEBUG] -- exited code 0 [2024-01-25 09:59:33.065] [DEBUG] -- admin@192.168.10.172 execute: ls /proc/23253 [2024-01-25 09:59:33.225] [DEBUG] -- exited code 0 [2024-01-25 09:59:33.226] [DEBUG] -- starting 192.168.10.168(192.168.10.168) observer [2024-01-25 09:59:33.227] [DEBUG] -- admin@192.168.10.168 set env LD_LIBRARY_PATH to '/home/admin/oceanbase-ce/lib:' [2024-01-25 09:59:33.227] [DEBUG] -- admin@192.168.10.168 execute: cd /home/admin/oceanbase-ce; /home/admin/oceanbase-ce/bin/observer -p 12881 [2024-01-25 09:59:33.307] [DEBUG] -- exited code 0 [2024-01-25 09:59:33.307] [DEBUG] -- admin@192.168.10.168 delete env LD_LIBRARY_PATH [2024-01-25 09:59:33.308] [DEBUG] -- starting 192.168.10.169(192.168.10.169) observer [2024-01-25 09:59:33.308] [DEBUG] -- admin@192.168.10.169 set env LD_LIBRARY_PATH to '/home/admin/oceanbase-ce/lib:' [2024-01-25 09:59:33.308] [DEBUG] -- admin@192.168.10.169 execute: cd /home/admin/oceanbase-ce; /home/admin/oceanbase-ce/bin/observer -p 12881 [2024-01-25 09:59:33.444] [DEBUG] -- exited code 0 [2024-01-25 09:59:33.444] [DEBUG] -- admin@192.168.10.169 delete env LD_LIBRARY_PATH [2024-01-25 09:59:33.519] [INFO] observer program health check [2024-01-25 09:59:36.523] [DEBUG] -- 192.168.10.168(192.168.10.168) program health check [2024-01-25 09:59:36.523] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:36.548] [DEBUG] -- exited code 0 [2024-01-25 09:59:36.549] [DEBUG] -- admin@192.168.10.168 execute: ls /proc/47855 [2024-01-25 09:59:36.613] [DEBUG] -- exited code 0 [2024-01-25 09:59:36.613] [DEBUG] -- 192.168.10.168(192.168.10.168) observer[pid: 47855] started [2024-01-25 09:59:36.614] [DEBUG] -- 192.168.10.169(192.168.10.169) program health check [2024-01-25 09:59:36.614] [DEBUG] -- admin@192.168.10.169 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:36.682] [DEBUG] -- exited code 0 [2024-01-25 09:59:36.683] [DEBUG] -- admin@192.168.10.169 execute: ls /proc/168568 [2024-01-25 09:59:36.793] [DEBUG] -- exited code 0 [2024-01-25 09:59:36.794] [DEBUG] -- 192.168.10.169(192.168.10.169) observer[pid: 168568] started [2024-01-25 09:59:36.794] [DEBUG] -- 192.168.10.172(192.168.10.172) program health check [2024-01-25 09:59:36.795] [DEBUG] -- admin@192.168.10.172 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-01-25 09:59:36.941] [DEBUG] -- exited code 0 [2024-01-25 09:59:36.941] [DEBUG] -- admin@192.168.10.172 execute: ls /proc/23253 [2024-01-25 09:59:37.120] [DEBUG] -- exited code 0 [2024-01-25 09:59:37.120] [DEBUG] -- 192.168.10.172(192.168.10.172) observer[pid: 23253] started [2024-01-25 09:59:37.170] [DEBUG] - sub start ref count to 0 [2024-01-25 09:59:37.171] [DEBUG] - export start [2024-01-25 09:59:37.171] [DEBUG] - Call oceanbase-ce-py_script_connect-3.1.0 for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:37.171] [DEBUG] - import connect [2024-01-25 09:59:37.223] [DEBUG] - add connect ref count to 1 [2024-01-25 09:59:37.224] [INFO] Connect to observer [2024-01-25 09:59:37.224] [DEBUG] -- connect 192.168.10.168 -P12881 -uroot -p [2024-01-25 09:59:37.486] [DEBUG] - sub connect ref count to 0 [2024-01-25 09:59:37.486] [DEBUG] - export connect [2024-01-25 09:59:37.486] [DEBUG] - Call obproxy-ce-py_script_start-3.1.0 for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:37.486] [DEBUG] - import start [2024-01-25 09:59:37.490] [DEBUG] - add start ref count to 1 [2024-01-25 09:59:37.496] [INFO] Start obproxy [2024-01-25 09:59:37.497] [DEBUG] -- admin@192.168.10.168 execute: ls /home/admin/obproxy/etc/obproxy_config.bin [2024-01-25 09:59:37.521] [DEBUG] -- exited code 0 [2024-01-25 09:59:37.522] [DEBUG] -- 192.168.10.168 port check [2024-01-25 09:59:37.522] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:37.583] [DEBUG] -- exited code 0 [2024-01-25 09:59:37.584] [DEBUG] -- admin@192.168.10.168 execute: ls /proc/41413/ [2024-01-25 09:59:37.647] [DEBUG] -- exited code 2, error output: [2024-01-25 09:59:37.647] [DEBUG] ls: cannot access /proc/41413/: No such file or directory [2024-01-25 09:59:37.648] [DEBUG] [2024-01-25 09:59:37.648] [DEBUG] -- starting 192.168.10.168 obproxy [2024-01-25 09:59:37.649] [DEBUG] -- admin@192.168.10.168 set env LD_LIBRARY_PATH to '/home/admin/obproxy/lib:' [2024-01-25 09:59:37.649] [DEBUG] -- admin@192.168.10.168 execute: cd /home/admin/obproxy; /home/admin/obproxy/bin/obproxy --listen_port 12883 [2024-01-25 09:59:37.733] [DEBUG] -- exited code 0 [2024-01-25 09:59:37.734] [DEBUG] -- admin@192.168.10.168 delete env LD_LIBRARY_PATH [2024-01-25 09:59:37.734] [DEBUG] -- admin@192.168.10.168 execute: ps -aux | grep -e '/home/admin/obproxy/bin/obproxy --listen_port 12883$' | grep -v grep | awk '{print $2}' > /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:37.835] [DEBUG] -- exited code 0 [2024-01-25 09:59:37.890] [INFO] obproxy program health check [2024-01-25 09:59:37.890] [DEBUG] -- 192.168.10.168 program health check [2024-01-25 09:59:37.891] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:37.912] [DEBUG] -- exited code 0 [2024-01-25 09:59:37.913] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3253' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:37.979] [DEBUG] -- exited code 0 [2024-01-25 09:59:37.979] [DEBUG] -- failed to start 192.168.10.168 obproxy, remaining retries: 19 [2024-01-25 09:59:38.981] [DEBUG] -- 192.168.10.168 program health check [2024-01-25 09:59:38.981] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:39.005] [DEBUG] -- exited code 0 [2024-01-25 09:59:39.006] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3253' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:39.077] [DEBUG] -- exited code 0 [2024-01-25 09:59:39.077] [DEBUG] -- 2891720 [2024-01-25 09:59:39.078] [DEBUG] [2024-01-25 09:59:39.078] [DEBUG] -- admin@192.168.10.168 execute: ls -l /proc/48542/fd/ |grep -E 'socket:\[(2891720)\]' [2024-01-25 09:59:39.141] [DEBUG] -- exited code 0 [2024-01-25 09:59:39.142] [DEBUG] -- admin@192.168.10.168 execute: pid=`cat /home/admin/obproxy/run/obproxyd-192.168.10.168-12883.pid` && ls /proc/$pid [2024-01-25 09:59:39.205] [DEBUG] -- exited code 2, error output: [2024-01-25 09:59:39.205] [DEBUG] ls: cannot access /proc/41307: No such file or directory [2024-01-25 09:59:39.205] [DEBUG] [2024-01-25 09:59:39.206] [DEBUG] -- admin@192.168.10.168 execute: echo 48542 > /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:39.265] [DEBUG] -- exited code 0 [2024-01-25 09:59:39.372] [DEBUG] - current remote_transporter RemoteTransporter.CLIENT [2024-01-25 09:59:39.373] [DEBUG] -- admin@192.168.10.168 execute: mkdir -p /home/admin/obproxy && rm -fr /home/admin/obproxy/obproxyd.sh [2024-01-25 09:59:39.394] [DEBUG] -- exited code 0 [2024-01-25 09:59:39.395] [DEBUG] -- send /home/admin/.obd/plugins/obproxy-ce/3.1.0/obproxyd.sh to /home/admin/obproxy/obproxyd.sh [2024-01-25 09:59:39.437] [DEBUG] - admin@192.168.10.168 execute: chmod 644 /home/admin/obproxy/obproxyd.sh [2024-01-25 09:59:39.459] [DEBUG] - exited code 0 [2024-01-25 09:59:39.459] [DEBUG] -- admin@192.168.10.168 execute: bash /home/admin/obproxy/obproxyd.sh /home/admin/obproxy 192.168.10.168 12883 [2024-01-25 09:59:39.520] [DEBUG] -- exited code 0 [2024-01-25 09:59:40.522] [DEBUG] -- 192.168.10.168 program health check [2024-01-25 09:59:40.522] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:40.547] [DEBUG] -- exited code 0 [2024-01-25 09:59:40.547] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3253' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:40.617] [DEBUG] -- exited code 0 [2024-01-25 09:59:40.618] [DEBUG] -- failed to start 192.168.10.168 obproxy, remaining retries: 17 [2024-01-25 09:59:41.619] [DEBUG] -- 192.168.10.168 program health check [2024-01-25 09:59:41.619] [DEBUG] -- admin@192.168.10.168 execute: cat /home/admin/obproxy/run/obproxy-192.168.10.168-12883.pid [2024-01-25 09:59:41.643] [DEBUG] -- exited code 0 [2024-01-25 09:59:41.643] [DEBUG] -- admin@192.168.10.168 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:3253' | awk -F' ' '{print $2}' | uniq [2024-01-25 09:59:41.714] [DEBUG] -- exited code 0 [2024-01-25 09:59:41.714] [DEBUG] -- 2962680 [2024-01-25 09:59:41.714] [DEBUG] [2024-01-25 09:59:41.715] [DEBUG] -- admin@192.168.10.168 execute: ls -l /proc/48959/fd/ |grep -E 'socket:\[(2962680)\]' [2024-01-25 09:59:41.781] [DEBUG] -- exited code 0 [2024-01-25 09:59:41.781] [DEBUG] -- admin@192.168.10.168 execute: pid=`cat /home/admin/obproxy/run/obproxyd-192.168.10.168-12883.pid` && ls /proc/$pid [2024-01-25 09:59:41.844] [DEBUG] -- exited code 0 [2024-01-25 09:59:41.845] [DEBUG] -- 192.168.10.168 obproxy[pid: 48959] started [2024-01-25 09:59:41.932] [DEBUG] - sub start ref count to 0 [2024-01-25 09:59:41.932] [DEBUG] - export start [2024-01-25 09:59:41.932] [DEBUG] - Call obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:41.932] [DEBUG] - import connect [2024-01-25 09:59:41.935] [DEBUG] - add connect ref count to 1 [2024-01-25 09:59:41.935] [INFO] Connect to obproxy [2024-01-25 09:59:41.937] [DEBUG] -- connect 192.168.10.168 -P12883 -uroot@proxysys -p [2024-01-25 09:59:42.068] [DEBUG] - sub connect ref count to 0 [2024-01-25 09:59:42.069] [DEBUG] - export connect [2024-01-25 09:59:42.069] [DEBUG] - Call oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-01-25 09:59:42.069] [DEBUG] - import display [2024-01-25 09:59:42.070] [DEBUG] - add display ref count to 1 [2024-01-25 09:59:42.071] [INFO] Wait for observer init [2024-01-25 09:59:42.072] [DEBUG] -- execute sql: select * from oceanbase.__all_server. args: None [2024-01-25 09:59:42.074] [DEBUG] -- OBD-5000: select * from oceanbase.__all_server execute failed [2024-01-25 09:59:42.076] [ERROR] Traceback (most recent call last): [2024-01-25 09:59:42.076] [ERROR] File "core.py", line 1564, in start_cluster [2024-01-25 09:59:42.076] [ERROR] File "core.py", line 1685, in _start_cluster [2024-01-25 09:59:42.077] [ERROR] File "core.py", line 182, in call_plugin [2024-01-25 09:59:42.077] [ERROR] File "_plugin.py", line 314, in __call__ [2024-01-25 09:59:42.077] [ERROR] File "_plugin.py", line 281, in _new_func [2024-01-25 09:59:42.077] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/display.py", line 37, in display [2024-01-25 09:59:42.077] [ERROR] servers = cursor.fetchall('select * from oceanbase.__all_server', raise_exception=True, exc_level='verbose') [2024-01-25 09:59:42.077] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:42.077] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 108, in fetchall [2024-01-25 09:59:42.077] [ERROR] return self.execute(sql, args=args, execute_func='fetchall', raise_exception=raise_exception, exc_level=exc_level, stdio=stdio) [2024-01-25 09:59:42.077] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:42.077] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 91, in execute [2024-01-25 09:59:42.077] [ERROR] self.cursor.execute(sql, args) [2024-01-25 09:59:42.077] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 148, in execute [2024-01-25 09:59:42.077] [ERROR] result = self._query(query) [2024-01-25 09:59:42.077] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 310, in _query [2024-01-25 09:59:42.078] [ERROR] conn.query(q) [2024-01-25 09:59:42.078] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 548, in query [2024-01-25 09:59:42.078] [ERROR] self._affected_rows = self._read_query_result(unbuffered=unbuffered) [2024-01-25 09:59:42.078] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 775, in _read_query_result [2024-01-25 09:59:42.078] [ERROR] result.read() [2024-01-25 09:59:42.078] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 1156, in read [2024-01-25 09:59:42.078] [ERROR] first_packet = self.connection._read_packet() [2024-01-25 09:59:42.078] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 725, in _read_packet [2024-01-25 09:59:42.078] [ERROR] packet.raise_for_error() [2024-01-25 09:59:42.078] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/protocol.py", line 221, in raise_for_error [2024-01-25 09:59:42.078] [ERROR] err.raise_mysql_exception(self._data) [2024-01-25 09:59:42.078] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/err.py", line 143, in raise_mysql_exception [2024-01-25 09:59:42.078] [ERROR] raise errorclass(errno, errval) [2024-01-25 09:59:42.078] [ERROR] pymysql.err.ProgrammingError: (1146, "Table 'oceanbase.__all_server' doesn't exist") [2024-01-25 09:59:42.079] [ERROR] [2024-01-25 09:59:45.080] [DEBUG] -- execute sql: select * from oceanbase.__all_server. args: None [2024-01-25 09:59:45.081] [DEBUG] -- OBD-5000: select * from oceanbase.__all_server execute failed [2024-01-25 09:59:45.082] [ERROR] Traceback (most recent call last): [2024-01-25 09:59:45.082] [ERROR] File "core.py", line 1564, in start_cluster [2024-01-25 09:59:45.082] [ERROR] File "core.py", line 1685, in _start_cluster [2024-01-25 09:59:45.082] [ERROR] File "core.py", line 182, in call_plugin [2024-01-25 09:59:45.082] [ERROR] File "_plugin.py", line 314, in __call__ [2024-01-25 09:59:45.083] [ERROR] File "_plugin.py", line 281, in _new_func [2024-01-25 09:59:45.083] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/display.py", line 37, in display [2024-01-25 09:59:45.083] [ERROR] servers = cursor.fetchall('select * from oceanbase.__all_server', raise_exception=True, exc_level='verbose') [2024-01-25 09:59:45.083] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:45.083] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 108, in fetchall [2024-01-25 09:59:45.083] [ERROR] return self.execute(sql, args=args, execute_func='fetchall', raise_exception=raise_exception, exc_level=exc_level, stdio=stdio) [2024-01-25 09:59:45.083] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:45.083] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 91, in execute [2024-01-25 09:59:45.083] [ERROR] self.cursor.execute(sql, args) [2024-01-25 09:59:45.083] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 148, in execute [2024-01-25 09:59:45.083] [ERROR] result = self._query(query) [2024-01-25 09:59:45.083] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 310, in _query [2024-01-25 09:59:45.083] [ERROR] conn.query(q) [2024-01-25 09:59:45.083] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 548, in query [2024-01-25 09:59:45.083] [ERROR] self._affected_rows = self._read_query_result(unbuffered=unbuffered) [2024-01-25 09:59:45.084] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 775, in _read_query_result [2024-01-25 09:59:45.084] [ERROR] result.read() [2024-01-25 09:59:45.084] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 1156, in read [2024-01-25 09:59:45.084] [ERROR] first_packet = self.connection._read_packet() [2024-01-25 09:59:45.084] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 725, in _read_packet [2024-01-25 09:59:45.084] [ERROR] packet.raise_for_error() [2024-01-25 09:59:45.084] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/protocol.py", line 221, in raise_for_error [2024-01-25 09:59:45.084] [ERROR] err.raise_mysql_exception(self._data) [2024-01-25 09:59:45.084] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/err.py", line 143, in raise_mysql_exception [2024-01-25 09:59:45.084] [ERROR] raise errorclass(errno, errval) [2024-01-25 09:59:45.084] [ERROR] pymysql.err.ProgrammingError: (1146, "Table 'oceanbase.__all_server' doesn't exist") [2024-01-25 09:59:45.084] [ERROR] [2024-01-25 09:59:48.087] [DEBUG] -- execute sql: select * from oceanbase.__all_server. args: None [2024-01-25 09:59:48.088] [DEBUG] -- OBD-5000: select * from oceanbase.__all_server execute failed [2024-01-25 09:59:48.088] [ERROR] Traceback (most recent call last): [2024-01-25 09:59:48.088] [ERROR] File "core.py", line 1564, in start_cluster [2024-01-25 09:59:48.088] [ERROR] File "core.py", line 1685, in _start_cluster [2024-01-25 09:59:48.089] [ERROR] File "core.py", line 182, in call_plugin [2024-01-25 09:59:48.089] [ERROR] File "_plugin.py", line 314, in __call__ [2024-01-25 09:59:48.089] [ERROR] File "_plugin.py", line 281, in _new_func [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/display.py", line 37, in display [2024-01-25 09:59:48.089] [ERROR] servers = cursor.fetchall('select * from oceanbase.__all_server', raise_exception=True, exc_level='verbose') [2024-01-25 09:59:48.089] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 108, in fetchall [2024-01-25 09:59:48.089] [ERROR] return self.execute(sql, args=args, execute_func='fetchall', raise_exception=raise_exception, exc_level=exc_level, stdio=stdio) [2024-01-25 09:59:48.089] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 91, in execute [2024-01-25 09:59:48.089] [ERROR] self.cursor.execute(sql, args) [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 148, in execute [2024-01-25 09:59:48.089] [ERROR] result = self._query(query) [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 310, in _query [2024-01-25 09:59:48.089] [ERROR] conn.query(q) [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 548, in query [2024-01-25 09:59:48.089] [ERROR] self._affected_rows = self._read_query_result(unbuffered=unbuffered) [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 775, in _read_query_result [2024-01-25 09:59:48.089] [ERROR] result.read() [2024-01-25 09:59:48.089] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 1156, in read [2024-01-25 09:59:48.090] [ERROR] first_packet = self.connection._read_packet() [2024-01-25 09:59:48.090] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 725, in _read_packet [2024-01-25 09:59:48.090] [ERROR] packet.raise_for_error() [2024-01-25 09:59:48.090] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/protocol.py", line 221, in raise_for_error [2024-01-25 09:59:48.090] [ERROR] err.raise_mysql_exception(self._data) [2024-01-25 09:59:48.090] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/err.py", line 143, in raise_mysql_exception [2024-01-25 09:59:48.090] [ERROR] raise errorclass(errno, errval) [2024-01-25 09:59:48.090] [ERROR] pymysql.err.ProgrammingError: (1146, "Table 'oceanbase.__all_server' doesn't exist") [2024-01-25 09:59:48.090] [ERROR] [2024-01-25 09:59:51.092] [DEBUG] -- execute sql: select * from oceanbase.__all_server. args: None [2024-01-25 09:59:51.092] [DEBUG] -- OBD-5000: select * from oceanbase.__all_server execute failed [2024-01-25 09:59:51.094] [ERROR] Traceback (most recent call last): [2024-01-25 09:59:51.094] [ERROR] File "core.py", line 1564, in start_cluster [2024-01-25 09:59:51.094] [ERROR] File "core.py", line 1685, in _start_cluster [2024-01-25 09:59:51.094] [ERROR] File "core.py", line 182, in call_plugin [2024-01-25 09:59:51.094] [ERROR] File "_plugin.py", line 314, in __call__ [2024-01-25 09:59:51.094] [ERROR] File "_plugin.py", line 281, in _new_func [2024-01-25 09:59:51.094] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/display.py", line 37, in display [2024-01-25 09:59:51.094] [ERROR] servers = cursor.fetchall('select * from oceanbase.__all_server', raise_exception=True, exc_level='verbose') [2024-01-25 09:59:51.094] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:51.094] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 108, in fetchall [2024-01-25 09:59:51.094] [ERROR] return self.execute(sql, args=args, execute_func='fetchall', raise_exception=raise_exception, exc_level=exc_level, stdio=stdio) [2024-01-25 09:59:51.095] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:51.095] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 91, in execute [2024-01-25 09:59:51.095] [ERROR] self.cursor.execute(sql, args) [2024-01-25 09:59:51.095] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 148, in execute [2024-01-25 09:59:51.095] [ERROR] result = self._query(query) [2024-01-25 09:59:51.095] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 310, in _query [2024-01-25 09:59:51.095] [ERROR] conn.query(q) [2024-01-25 09:59:51.095] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 548, in query [2024-01-25 09:59:51.095] [ERROR] self._affected_rows = self._read_query_result(unbuffered=unbuffered) [2024-01-25 09:59:51.095] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 775, in _read_query_result [2024-01-25 09:59:51.095] [ERROR] result.read() [2024-01-25 09:59:51.095] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 1156, in read [2024-01-25 09:59:51.095] [ERROR] first_packet = self.connection._read_packet() [2024-01-25 09:59:51.096] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 725, in _read_packet [2024-01-25 09:59:51.096] [ERROR] packet.raise_for_error() [2024-01-25 09:59:51.096] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/protocol.py", line 221, in raise_for_error [2024-01-25 09:59:51.096] [ERROR] err.raise_mysql_exception(self._data) [2024-01-25 09:59:51.096] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/err.py", line 143, in raise_mysql_exception [2024-01-25 09:59:51.096] [ERROR] raise errorclass(errno, errval) [2024-01-25 09:59:51.096] [ERROR] pymysql.err.ProgrammingError: (1146, "Table 'oceanbase.__all_server' doesn't exist") [2024-01-25 09:59:51.096] [ERROR] [2024-01-25 09:59:54.099] [DEBUG] -- execute sql: select * from oceanbase.__all_server. args: None [2024-01-25 09:59:54.100] [DEBUG] -- OBD-5000: select * from oceanbase.__all_server execute failed [2024-01-25 09:59:54.101] [ERROR] Traceback (most recent call last): [2024-01-25 09:59:54.101] [ERROR] File "core.py", line 1564, in start_cluster [2024-01-25 09:59:54.101] [ERROR] File "core.py", line 1685, in _start_cluster [2024-01-25 09:59:54.101] [ERROR] File "core.py", line 182, in call_plugin [2024-01-25 09:59:54.101] [ERROR] File "_plugin.py", line 314, in __call__ [2024-01-25 09:59:54.101] [ERROR] File "_plugin.py", line 281, in _new_func [2024-01-25 09:59:54.101] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/display.py", line 37, in display [2024-01-25 09:59:54.101] [ERROR] servers = cursor.fetchall('select * from oceanbase.__all_server', raise_exception=True, exc_level='verbose') [2024-01-25 09:59:54.101] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:54.101] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 108, in fetchall [2024-01-25 09:59:54.101] [ERROR] return self.execute(sql, args=args, execute_func='fetchall', raise_exception=raise_exception, exc_level=exc_level, stdio=stdio) [2024-01-25 09:59:54.101] [ERROR] File "_stdio.py", line 865, in func_wrapper [2024-01-25 09:59:54.101] [ERROR] File "/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py", line 91, in execute [2024-01-25 09:59:54.101] [ERROR] self.cursor.execute(sql, args) [2024-01-25 09:59:54.101] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 148, in execute [2024-01-25 09:59:54.102] [ERROR] result = self._query(query) [2024-01-25 09:59:54.102] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py", line 310, in _query [2024-01-25 09:59:54.102] [ERROR] conn.query(q) [2024-01-25 09:59:54.102] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 548, in query [2024-01-25 09:59:54.102] [ERROR] self._affected_rows = self._read_query_result(unbuffered=unbuffered) [2024-01-25 09:59:54.102] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 775, in _read_query_result [2024-01-25 09:59:54.102] [ERROR] result.read() [2024-01-25 09:59:54.102] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 1156, in read [2024-01-25 09:59:54.102] [ERROR] first_packet = self.connection._read_packet() [2024-01-25 09:59:54.102] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 692, in _read_packet [2024-01-25 09:59:54.102] [ERROR] packet_header = self._read_bytes(4) [2024-01-25 09:59:54.102] [ERROR] File "/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py", line 748, in _read_bytes [2024-01-25 09:59:54.102] [ERROR] raise err.OperationalError( [2024-01-25 09:59:54.102] [ERROR] pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query') [2024-01-25 09:59:54.102] [ERROR] [2024-01-25 09:59:54.189] [DEBUG] - sub display ref count to 0 [2024-01-25 09:59:54.190] [DEBUG] - export display [2024-01-25 09:59:54.190] [DEBUG] - Call obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-01-25 09:59:54.190] [DEBUG] - import display [2024-01-25 09:59:54.191] [DEBUG] - add display ref count to 1 [2024-01-25 09:59:54.192] [DEBUG] -- execute sql: show proxyconfig like "%port". args: None [2024-01-25 09:59:54.206] [INFO] +---------------------------------------------------+ [2024-01-25 09:59:54.206] [INFO] | obproxy | [2024-01-25 09:59:54.207] [INFO] +----------------+-------+-----------------+--------+ [2024-01-25 09:59:54.207] [INFO] | ip | port | prometheus_port | status | [2024-01-25 09:59:54.207] [INFO] +----------------+-------+-----------------+--------+ [2024-01-25 09:59:54.207] [INFO] | 192.168.10.168 | 12883 | 12884 | active | [2024-01-25 09:59:54.207] [INFO] +----------------+-------+-----------------+--------+ [2024-01-25 09:59:54.208] [INFO] obclient -h192.168.10.168 -P12883 -uroot -Doceanbase -A [2024-01-25 09:59:54.208] [DEBUG] - sub display ref count to 0 [2024-01-25 09:59:54.208] [DEBUG] - export display [2024-01-25 09:59:54.209] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes . [2024-01-25 09:59:54.209] [INFO] Trace ID: 5f4f661a-bb25-11ee-9bbf-246e9672bf10 [2024-01-25 09:59:54.209] [INFO] If you want to view detailed obd logs, please run: obd display-trace 5f4f661a-bb25-11ee-9bbf-246e9672bf10 [2024-01-25 09:59:54.209] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 1 [2024-01-25 09:59:54.209] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 0 [2024-01-25 09:59:54.209] [DEBUG] - unlock /home/admin/.obd/lock/mirror_and_repo [2024-01-25 09:59:54.209] [DEBUG] - exclusive lock /home/admin/.obd/lock/deploy_obtest release, count 0 [2024-01-25 09:59:54.209] [DEBUG] - unlock /home/admin/.obd/lock/deploy_obtest [2024-01-25 09:59:54.210] [DEBUG] - share lock /home/admin/.obd/lock/global release, count 0 [2024-01-25 09:59:54.210] [DEBUG] - unlock /home/admin/.obd/lock/global