[2024-04-01 01:48:24.404] [DEBUG] - cmd: ['OCP'] [2024-04-01 01:48:24.404] [DEBUG] - opts: {'servers': None, 'components': None, 'force_delete': None, 'strict_check': None, 'without_parameter': None} [2024-04-01 01:48:24.405] [DEBUG] - mkdir /root/.obd/lock/ [2024-04-01 01:48:24.405] [DEBUG] - unknown lock mode [2024-04-01 01:48:24.406] [DEBUG] - try to get share lock /root/.obd/lock/global [2024-04-01 01:48:24.406] [DEBUG] - share lock `/root/.obd/lock/global`, count 1 [2024-04-01 01:48:24.406] [DEBUG] - Get Deploy by name [2024-04-01 01:48:24.407] [DEBUG] - mkdir /root/.obd/cluster/ [2024-04-01 01:48:24.407] [DEBUG] - mkdir /root/.obd/config_parser/ [2024-04-01 01:48:24.407] [DEBUG] - try to get exclusive lock /root/.obd/lock/deploy_OCP [2024-04-01 01:48:24.408] [DEBUG] - exclusive lock `/root/.obd/lock/deploy_OCP`, count 1 [2024-04-01 01:48:24.421] [DEBUG] - Deploy status judge [2024-04-01 01:48:24.422] [INFO] Get local repositories [2024-04-01 01:48:24.423] [DEBUG] - mkdir /root/.obd/repository [2024-04-01 01:48:24.423] [DEBUG] - Get local repository oceanbase-ce-4.2.2.1-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.424] [DEBUG] - Search repository oceanbase-ce version: 4.2.2.1, tag: 3e30b9c0f183a5ef764ab754d5e517a671d96b38, release: None, package_hash: None [2024-04-01 01:48:24.424] [DEBUG] - try to get share lock /root/.obd/lock/mirror_and_repo [2024-04-01 01:48:24.424] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 1 [2024-04-01 01:48:24.424] [DEBUG] - mkdir /root/.obd/repository/oceanbase-ce [2024-04-01 01:48:24.429] [DEBUG] - Found repository oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.429] [DEBUG] - Get local repository obproxy-ce-4.2.3.0-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.429] [DEBUG] - Search repository obproxy-ce version: 4.2.3.0, tag: 0490ebc04220def8d25cb9cac9ac61a4efa6d639, release: None, package_hash: None [2024-04-01 01:48:24.430] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 2 [2024-04-01 01:48:24.430] [DEBUG] - mkdir /root/.obd/repository/obproxy-ce [2024-04-01 01:48:24.433] [DEBUG] - Found repository obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.434] [DEBUG] - Get local repository ocp-server-ce-4.2.1-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.434] [DEBUG] - Search repository ocp-server-ce version: 4.2.1, tag: 58cf72891d75a2fa7c754bafc42d336525baf0b5, release: None, package_hash: None [2024-04-01 01:48:24.434] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 3 [2024-04-01 01:48:24.434] [DEBUG] - mkdir /root/.obd/repository/ocp-server-ce [2024-04-01 01:48:24.438] [DEBUG] - Found repository ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.554] [DEBUG] - Get deploy config [2024-04-01 01:48:24.608] [INFO] Search plugins [2024-04-01 01:48:24.609] [DEBUG] - Searching start_check plugin for components ... [2024-04-01 01:48:24.610] [DEBUG] - Searching start_check plugin for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.610] [DEBUG] - mkdir /root/.obd/plugins [2024-04-01 01:48:24.612] [DEBUG] - Found for oceanbase-ce-py_script_start_check-4.2.2.0 for oceanbase-ce-4.2.2.1 [2024-04-01 01:48:24.612] [DEBUG] - Searching start_check plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.612] [DEBUG] - Found for obproxy-ce-py_script_start_check-3.1.0 for obproxy-ce-4.2.3.0 [2024-04-01 01:48:24.612] [DEBUG] - Searching start_check plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.613] [DEBUG] - Found for ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1 [2024-04-01 01:48:24.613] [DEBUG] - Searching create_tenant plugin for components ... [2024-04-01 01:48:24.613] [DEBUG] - Searching create_tenant plugin for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.614] [DEBUG] - Found for oceanbase-ce-py_script_create_tenant-4.2.0.0 for oceanbase-ce-4.2.2.1 [2024-04-01 01:48:24.614] [DEBUG] - Searching create_tenant plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.614] [DEBUG] - No such create_tenant plugin for obproxy-ce-4.2.3.0 [2024-04-01 01:48:24.614] [DEBUG] - Searching create_tenant plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.615] [DEBUG] - No such create_tenant plugin for ocp-server-ce-4.2.1 [2024-04-01 01:48:24.615] [DEBUG] - Searching start plugin for components ... [2024-04-01 01:48:24.615] [DEBUG] - Searching start plugin for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.616] [DEBUG] - Found for oceanbase-ce-py_script_start-4.2.2.0 for oceanbase-ce-4.2.2.1 [2024-04-01 01:48:24.616] [DEBUG] - Searching start plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.617] [DEBUG] - Found for obproxy-ce-py_script_start-3.1.0 for obproxy-ce-4.2.3.0 [2024-04-01 01:48:24.617] [DEBUG] - Searching start plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.617] [DEBUG] - Found for ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1 [2024-04-01 01:48:24.617] [DEBUG] - Searching connect plugin for components ... [2024-04-01 01:48:24.617] [DEBUG] - Searching connect plugin for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.618] [DEBUG] - Found for oceanbase-ce-py_script_connect-4.2.2.0 for oceanbase-ce-4.2.2.1 [2024-04-01 01:48:24.618] [DEBUG] - Searching connect plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.618] [DEBUG] - Found for obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.2.3.0 [2024-04-01 01:48:24.619] [DEBUG] - Searching connect plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.619] [DEBUG] - Found for ocp-server-ce-py_script_connect-4.2.1 for ocp-server-ce-4.2.1 [2024-04-01 01:48:24.619] [DEBUG] - Searching bootstrap plugin for components ... [2024-04-01 01:48:24.619] [DEBUG] - Searching bootstrap plugin for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.620] [DEBUG] - Found for oceanbase-ce-py_script_bootstrap-4.2.2.0 for oceanbase-ce-4.2.2.1 [2024-04-01 01:48:24.620] [DEBUG] - Searching bootstrap plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.621] [DEBUG] - Found for obproxy-ce-py_script_bootstrap-3.1.0 for obproxy-ce-4.2.3.0 [2024-04-01 01:48:24.621] [DEBUG] - Searching bootstrap plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.621] [DEBUG] - Found for ocp-server-ce-py_script_bootstrap-4.2.1 for ocp-server-ce-4.2.1 [2024-04-01 01:48:24.621] [DEBUG] - Searching display plugin for components ... [2024-04-01 01:48:24.621] [DEBUG] - Searching display plugin for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.622] [DEBUG] - Found for oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.2.2.1 [2024-04-01 01:48:24.622] [DEBUG] - Searching display plugin for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.623] [DEBUG] - Found for obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.2.3.0 [2024-04-01 01:48:24.623] [DEBUG] - Searching display plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.623] [DEBUG] - Found for ocp-server-ce-py_script_display-4.2.1 for ocp-server-ce-4.2.1 [2024-04-01 01:48:24.741] [INFO] Load cluster param plugin [2024-04-01 01:48:24.742] [DEBUG] - Get local repository oceanbase-ce-4.2.2.1-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:24.742] [DEBUG] - Get local repository obproxy-ce-4.2.3.0-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:24.742] [DEBUG] - Get local repository ocp-server-ce-4.2.1-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:24.742] [DEBUG] - Searching param plugin for components ... [2024-04-01 01:48:24.743] [DEBUG] - Search param plugin for oceanbase-ce [2024-04-01 01:48:24.743] [DEBUG] - Found for oceanbase-ce-param-4.2.2.0 for oceanbase-ce-4.2.2.1 [2024-04-01 01:48:24.744] [DEBUG] - Applying oceanbase-ce-param-4.2.2.0 for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:25.840] [DEBUG] - Search param plugin for obproxy-ce [2024-04-01 01:48:25.841] [DEBUG] - Found for obproxy-ce-param-3.1.0 for obproxy-ce-4.2.3.0 [2024-04-01 01:48:25.841] [DEBUG] - Applying obproxy-ce-param-3.1.0 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:26.094] [DEBUG] - Search param plugin for ocp-server-ce [2024-04-01 01:48:26.095] [DEBUG] - Found for ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1 [2024-04-01 01:48:26.095] [DEBUG] - Applying ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:26.325] [INFO] Open ssh connection [2024-04-01 01:48:26.326] [DEBUG] - host: 165.3.87.9, port: 22, user: oceanbase, password: w501g8la. [2024-04-01 01:48:34.547] [DEBUG] - Call oceanbase-ce-py_script_start_check-4.2.2.0 for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:34.547] [DEBUG] - import start_check [2024-04-01 01:48:34.565] [DEBUG] - add start_check ref count to 1 [2024-04-01 01:48:34.566] [INFO] Check before start observer [2024-04-01 01:48:34.570] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /data/1/clog/tenant_1/ [2024-04-01 01:48:34.771] [DEBUG] -- exited code 0 [2024-04-01 01:48:34.772] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/run/observer.pid [2024-04-01 01:48:34.819] [DEBUG] -- exited code 0 [2024-04-01 01:48:34.820] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/21716 [2024-04-01 01:48:34.869] [DEBUG] -- exited code 0 [2024-04-01 01:48:34.870] [DEBUG] -- 165.3.87.9 is runnning, skip [2024-04-01 01:48:34.871] [DEBUG] -- oceanbase@165.3.87.9 execute: [ -w /tmp/ ] || [ -w /tmp/obshell ] [2024-04-01 01:48:34.915] [DEBUG] -- exited code 0 [2024-04-01 01:48:34.916] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /proc/sys/fs/aio-max-nr /proc/sys/fs/aio-nr [2024-04-01 01:48:34.964] [DEBUG] -- exited code 0 [2024-04-01 01:48:34.964] [DEBUG] -- oceanbase@165.3.87.9 execute: ulimit -a [2024-04-01 01:48:35.011] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.012] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /proc/meminfo [2024-04-01 01:48:35.060] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.062] [DEBUG] -- oceanbase@165.3.87.9 execute: df --block-size=1024 [2024-04-01 01:48:35.110] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.111] [DEBUG] -- get disk info for path /, total: 150312300544 avail: 6025928704 [2024-04-01 01:48:35.112] [DEBUG] -- get disk info for path /dev, total: 8304988160 avail: 8304988160 [2024-04-01 01:48:35.112] [DEBUG] -- get disk info for path /dev/shm, total: 8328392704 avail: 8328392704 [2024-04-01 01:48:35.112] [DEBUG] -- get disk info for path /run, total: 8328392704 avail: 8294223872 [2024-04-01 01:48:35.112] [DEBUG] -- get disk info for path /sys/fs/cgroup, total: 8328392704 avail: 8328392704 [2024-04-01 01:48:35.113] [DEBUG] -- get disk info for path /run/user/1000, total: 1665679360 avail: 1665679360 [2024-04-01 01:48:35.113] [DEBUG] -- get disk info for path /run/user/0, total: 1665679360 avail: 1665679360 [2024-04-01 01:48:35.113] [DEBUG] -- disk: {'/': {'total': 150312300544, 'avail': 6025928704, 'need': 0}, '/dev': {'total': 8304988160, 'avail': 8304988160, 'need': 0}, '/dev/shm': {'total': 8328392704, 'avail': 8328392704, 'need': 0}, '/run': {'total': 8328392704, 'avail': 8294223872, 'need': 0}, '/sys/fs/cgroup': {'total': 8328392704, 'avail': 8328392704, 'need': 0}, '/run/user/1000': {'total': 1665679360, 'avail': 1665679360, 'need': 0}, '/run/user/0': {'total': 1665679360, 'avail': 1665679360, 'need': 0}} [2024-04-01 01:48:35.114] [DEBUG] -- oceanbase@165.3.87.9 execute: date +%s%N [2024-04-01 01:48:35.159] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.159] [DEBUG] -- 165.3.87.9 time delta -2.561279296875 [2024-04-01 01:48:35.221] [DEBUG] - sub start_check ref count to 0 [2024-04-01 01:48:35.221] [DEBUG] - export start_check [2024-04-01 01:48:35.222] [DEBUG] - Call obproxy-ce-py_script_start_check-3.1.0 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:35.222] [DEBUG] - import start_check [2024-04-01 01:48:35.226] [DEBUG] - add start_check ref count to 1 [2024-04-01 01:48:35.227] [INFO] Check before start obproxy [2024-04-01 01:48:35.229] [WARNING] OBD-4521: The config observer_sys_password in obproxy-ce did not take effect, please config it in oceanbase-ce [2024-04-01 01:48:35.230] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/obproxy/run/obproxy-165.3.87.9-2883.pid [2024-04-01 01:48:35.239] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.240] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/22416/fd [2024-04-01 01:48:35.289] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.290] [DEBUG] -- 165.3.87.9 is runnning, skip [2024-04-01 01:48:35.361] [INFO] [WARN] OBD-4521: The config observer_sys_password in obproxy-ce did not take effect, please config it in oceanbase-ce [2024-04-01 01:48:35.361] [INFO] [2024-04-01 01:48:35.362] [DEBUG] - sub start_check ref count to 0 [2024-04-01 01:48:35.362] [DEBUG] - export start_check [2024-04-01 01:48:35.362] [DEBUG] - Call ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:35.362] [DEBUG] - import start_check [2024-04-01 01:48:35.373] [DEBUG] - add start_check ref count to 1 [2024-04-01 01:48:35.374] [INFO] Check before start ocp-server [2024-04-01 01:48:35.378] [DEBUG] -- oceanbase version check [2024-04-01 01:48:35.378] [DEBUG] -- oceanbase@165.3.87.9 execute: sudo -n true [2024-04-01 01:48:35.410] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.411] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/ocp/run/ocp-server.pid [2024-04-01 01:48:35.461] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.461] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/23585 [2024-04-01 01:48:35.511] [DEBUG] -- exited code 2, error output: [2024-04-01 01:48:35.511] [DEBUG] ls: cannot access /proc/23585: No such file or directory [2024-04-01 01:48:35.511] [DEBUG] [2024-04-01 01:48:35.512] [DEBUG] -- user check [2024-04-01 01:48:35.512] [DEBUG] -- port check [2024-04-01 01:48:35.512] [DEBUG] -- oceanbase@165.3.87.9 execute: bash -c 'cat /proc/net/{udp*,tcp*}' | awk -F' ' '{if($4=="0A") print $2,$4,$10}' | grep ':1F90' | awk -F' ' '{print $3}' | uniq [2024-04-01 01:48:35.566] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.567] [DEBUG] -- java check [2024-04-01 01:48:35.567] [DEBUG] -- oceanbase@165.3.87.9 execute: java -version [2024-04-01 01:48:35.735] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.736] [DEBUG] -- java version [2024-04-01 01:48:35.736] [DEBUG] -- java_major_version 1.8.0 [2024-04-01 01:48:35.736] [DEBUG] -- java_update_version 402 [2024-04-01 01:48:35.737] [DEBUG] -- clockdiff check [2024-04-01 01:48:35.737] [DEBUG] -- oceanbase@165.3.87.9 execute: which clockdiff [2024-04-01 01:48:35.782] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.783] [DEBUG] -- memory check [2024-04-01 01:48:35.783] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /proc/meminfo [2024-04-01 01:48:35.829] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.830] [DEBUG] -- disk check [2024-04-01 01:48:35.831] [DEBUG] -- oceanbase@165.3.87.9 execute: df --block-size=1024 [2024-04-01 01:48:35.879] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.879] [DEBUG] -- get disk info for path /, total: 150312300544 avail: 6025908224 [2024-04-01 01:48:35.880] [DEBUG] -- get disk info for path /dev, total: 8304988160 avail: 8304988160 [2024-04-01 01:48:35.880] [DEBUG] -- get disk info for path /dev/shm, total: 8328392704 avail: 8328392704 [2024-04-01 01:48:35.880] [DEBUG] -- get disk info for path /run, total: 8328392704 avail: 8294223872 [2024-04-01 01:48:35.880] [DEBUG] -- get disk info for path /sys/fs/cgroup, total: 8328392704 avail: 8328392704 [2024-04-01 01:48:35.880] [DEBUG] -- get disk info for path /run/user/1000, total: 1665679360 avail: 1665679360 [2024-04-01 01:48:35.880] [DEBUG] -- get disk info for path /run/user/0, total: 1665679360 avail: 1665679360 [2024-04-01 01:48:35.881] [DEBUG] -- oceanbase@165.3.87.9 execute: df --block-size=1024 /home/oceanbase/logs [2024-04-01 01:48:35.927] [DEBUG] -- exited code 0 [2024-04-01 01:48:35.927] [DEBUG] -- get disk info for path /, total: 150312300544 avail: 6025908224 [2024-04-01 01:48:36.031] [DEBUG] - sub start_check ref count to 0 [2024-04-01 01:48:36.032] [DEBUG] - export start_check [2024-04-01 01:48:36.032] [DEBUG] - Call oceanbase-ce-py_script_start-4.2.2.0 for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:36.032] [DEBUG] - import start [2024-04-01 01:48:36.037] [DEBUG] - add start ref count to 1 [2024-04-01 01:48:36.038] [INFO] Start observer [2024-04-01 01:48:36.038] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /data/1/clog/tenant_1/ [2024-04-01 01:48:36.048] [DEBUG] -- exited code 0 [2024-04-01 01:48:36.049] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/run/observer.pid [2024-04-01 01:48:36.096] [DEBUG] -- exited code 0 [2024-04-01 01:48:36.097] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/21716 [2024-04-01 01:48:36.146] [DEBUG] -- exited code 0 [2024-04-01 01:48:36.170] [DEBUG] -- start_obshell: True [2024-04-01 01:48:36.171] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/run/obshell.pid [2024-04-01 01:48:36.194] [DEBUG] -- exited code 0 [2024-04-01 01:48:36.195] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/22264 [2024-04-01 01:48:36.244] [DEBUG] -- exited code 0 [2024-04-01 01:48:36.245] [DEBUG] -- 165.3.87.9 obshell[pid: 22264] started [2024-04-01 01:48:36.246] [INFO] observer program health check [2024-04-01 01:48:39.250] [DEBUG] -- 165.3.87.9 program health check [2024-04-01 01:48:39.250] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/run/observer.pid [2024-04-01 01:48:39.259] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.260] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/21716 [2024-04-01 01:48:39.309] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.310] [DEBUG] -- 165.3.87.9 observer[pid: 21716] started [2024-04-01 01:48:39.380] [INFO] obshell program health check [2024-04-01 01:48:39.381] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/run/obshell.pid [2024-04-01 01:48:39.390] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.391] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/22264 [2024-04-01 01:48:39.440] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.441] [DEBUG] -- 165.3.87.9 obshell[pid: 22264] started [2024-04-01 01:48:39.512] [DEBUG] -- need_bootstrap: False [2024-04-01 01:48:39.513] [DEBUG] - sub start ref count to 0 [2024-04-01 01:48:39.513] [DEBUG] - export start [2024-04-01 01:48:39.513] [DEBUG] - Call oceanbase-ce-py_script_connect-4.2.2.0 for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:39.513] [DEBUG] - import connect [2024-04-01 01:48:39.682] [DEBUG] - add connect ref count to 1 [2024-04-01 01:48:39.682] [DEBUG] -- connect obshell (165.3.87.9:2886) [2024-04-01 01:48:39.683] [INFO] Connect to observer [2024-04-01 01:48:39.685] [DEBUG] -- connect 165.3.87.9 -P2881 -uroot -pv/JgI&D95 [2024-04-01 01:48:39.688] [DEBUG] -- execute sql: select 1. args: None [2024-04-01 01:48:39.815] [DEBUG] - sub connect ref count to 0 [2024-04-01 01:48:39.815] [DEBUG] - export connect [2024-04-01 01:48:39.816] [DEBUG] - Call obproxy-ce-py_script_start-3.1.0 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:39.816] [DEBUG] - import start [2024-04-01 01:48:39.821] [DEBUG] - add start ref count to 1 [2024-04-01 01:48:39.825] [INFO] Start obproxy [2024-04-01 01:48:39.825] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /home/oceanbase/obproxy/etc/obproxy_config.bin [2024-04-01 01:48:39.835] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.836] [DEBUG] -- 165.3.87.9 port check [2024-04-01 01:48:39.836] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/obproxy/run/obproxy-165.3.87.9-2883.pid [2024-04-01 01:48:39.883] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.884] [DEBUG] -- oceanbase@165.3.87.9 execute: ls /proc/22416/ [2024-04-01 01:48:39.934] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.935] [DEBUG] -- oceanbase@165.3.87.9 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:0B43' | awk -F' ' '{print $2}' | uniq [2024-04-01 01:48:39.988] [DEBUG] -- exited code 0 [2024-04-01 01:48:39.989] [DEBUG] -- 409271 [2024-04-01 01:48:39.990] [DEBUG] [2024-04-01 01:48:39.990] [DEBUG] -- oceanbase@165.3.87.9 execute: ls -l /proc/22416/fd/ |grep -E 'socket:\[(409271)\]' [2024-04-01 01:48:40.041] [DEBUG] -- exited code 0 [2024-04-01 01:48:40.089] [INFO] obproxy program health check [2024-04-01 01:48:40.089] [DEBUG] -- 165.3.87.9 program health check [2024-04-01 01:48:40.090] [DEBUG] -- oceanbase@165.3.87.9 execute: cat /home/oceanbase/obproxy/run/obproxy-165.3.87.9-2883.pid [2024-04-01 01:48:40.098] [DEBUG] -- exited code 0 [2024-04-01 01:48:40.099] [DEBUG] -- oceanbase@165.3.87.9 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:0B43' | awk -F' ' '{print $2}' | uniq [2024-04-01 01:48:40.152] [DEBUG] -- exited code 0 [2024-04-01 01:48:40.152] [DEBUG] -- 409271 [2024-04-01 01:48:40.153] [DEBUG] [2024-04-01 01:48:40.153] [DEBUG] -- oceanbase@165.3.87.9 execute: ls -l /proc/22416/fd/ |grep -E 'socket:\[(409271)\]' [2024-04-01 01:48:40.205] [DEBUG] -- exited code 0 [2024-04-01 01:48:40.206] [DEBUG] -- oceanbase@165.3.87.9 execute: pid=`cat /home/oceanbase/obproxy/run/obproxyd-165.3.87.9-2883.pid` && ls /proc/$pid [2024-04-01 01:48:40.257] [DEBUG] -- exited code 0 [2024-04-01 01:48:40.257] [DEBUG] -- 165.3.87.9 obproxy[pid: 22416] started [2024-04-01 01:48:40.351] [DEBUG] - sub start ref count to 0 [2024-04-01 01:48:40.352] [DEBUG] - export start [2024-04-01 01:48:40.352] [DEBUG] - Call obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:40.352] [DEBUG] - import connect [2024-04-01 01:48:40.356] [DEBUG] - add connect ref count to 1 [2024-04-01 01:48:40.356] [INFO] Connect to obproxy [2024-04-01 01:48:40.358] [DEBUG] -- connect 165.3.87.9 -P2883 -uroot@proxysys -p5QOQuVLt2q [2024-04-01 01:48:40.489] [DEBUG] - sub connect ref count to 0 [2024-04-01 01:48:40.489] [DEBUG] - export connect [2024-04-01 01:48:40.490] [INFO] Initialize obproxy-ce [2024-04-01 01:48:40.491] [DEBUG] - Call obproxy-ce-py_script_bootstrap-3.1.0 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:40.491] [DEBUG] - import bootstrap [2024-04-01 01:48:40.493] [DEBUG] - add bootstrap ref count to 1 [2024-04-01 01:48:40.493] [DEBUG] -- execute sql: alter proxyconfig set observer_sys_password = %s. args: ['cQFIELC6CC'] [2024-04-01 01:48:40.507] [DEBUG] - sub bootstrap ref count to 0 [2024-04-01 01:48:40.508] [DEBUG] - export bootstrap [2024-04-01 01:48:40.623] [DEBUG] - Call ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-04-01 01:48:40.623] [DEBUG] - import start [2024-04-01 01:48:40.632] [DEBUG] - add start ref count to 1 [2024-04-01 01:48:40.632] [DEBUG] -- metadb connect check [2024-04-01 01:48:40.634] [INFO] Start ocp-server [2024-04-01 01:48:40.635] [DEBUG] -- cp rpm & pos [2024-04-01 01:48:40.692] [DEBUG] - current remote_transporter RemoteTransporter.CLIENT [2024-04-01 01:48:40.693] [DEBUG] -- oceanbase@165.3.87.9 execute: mkdir -p /home/oceanbase/ocp/ocp-server/lib && rm -fr /home/oceanbase/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-04-01 01:48:40.723] [DEBUG] -- exited code 0 [2024-04-01 01:48:40.723] [DEBUG] -- send /root/.obd/mirror/local/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm to /home/oceanbase/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-04-01 01:48:44.200] [DEBUG] - oceanbase@165.3.87.9 execute: chmod 644 /home/oceanbase/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-04-01 01:48:44.211] [DEBUG] - exited code 0 [2024-04-01 01:48:44.212] [DEBUG] -- oceanbase@165.3.87.9 execute: mkdir -p /home/oceanbase/software && rm -fr /home/oceanbase/software/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-04-01 01:48:44.284] [DEBUG] -- exited code 0 [2024-04-01 01:48:44.284] [DEBUG] -- send /root/.obd/mirror/local/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm to /home/oceanbase/software/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-04-01 01:48:47.671] [DEBUG] - oceanbase@165.3.87.9 execute: chmod 644 /home/oceanbase/software/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-04-01 01:48:47.680] [DEBUG] - exited code 0 [2024-04-01 01:48:47.681] [DEBUG] -- oceanbase@165.3.87.9 execute: mkdir -p /home/oceanbase/ocp/ocp-server/lib && rm -fr /home/oceanbase/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-04-01 01:48:47.771] [DEBUG] -- exited code 0 [2024-04-01 01:48:47.771] [DEBUG] -- send /root/.obd/mirror/local/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm to /home/oceanbase/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-04-01 01:48:51.304] [ERROR] ocp-server-ce-py_script_start-4.2.1 RuntimeError: Server connection dropped: [2024-04-01 01:48:51.304] [ERROR] Traceback (most recent call last): [2024-04-01 01:48:51.304] [ERROR] File "paramiko/sftp_client.py", line 716, in putfo [2024-04-01 01:48:51.304] [ERROR] File "paramiko/sftp_client.py", line 679, in _transfer_with_callback [2024-04-01 01:48:51.304] [ERROR] File "paramiko/file.py", line 405, in write [2024-04-01 01:48:51.304] [ERROR] File "paramiko/file.py", line 523, in _write_all [2024-04-01 01:48:51.304] [ERROR] File "paramiko/sftp_file.py", line 195, in _write [2024-04-01 01:48:51.304] [ERROR] File "paramiko/sftp_client.py", line 846, in _async_request [2024-04-01 01:48:51.305] [ERROR] File "paramiko/sftp.py", line 198, in _send_packet [2024-04-01 01:48:51.305] [ERROR] File "paramiko/sftp.py", line 162, in _write_all [2024-04-01 01:48:51.305] [ERROR] File "paramiko/channel.py", line 801, in send [2024-04-01 01:48:51.305] [ERROR] File "paramiko/channel.py", line 1208, in _send [2024-04-01 01:48:51.305] [ERROR] File "paramiko/transport.py", line 1917, in _send_user_message [2024-04-01 01:48:51.305] [ERROR] File "paramiko/transport.py", line 1893, in _send_message [2024-04-01 01:48:51.305] [ERROR] File "paramiko/packet.py", line 425, in send_message [2024-04-01 01:48:51.305] [ERROR] File "paramiko/packet.py", line 47, in compute_hmac [2024-04-01 01:48:51.305] [ERROR] File "hmac.py", line 88, in __init__ [2024-04-01 01:48:51.305] [ERROR] File "hmac.py", line 96, in update [2024-04-01 01:48:51.306] [ERROR] KeyboardInterrupt [2024-04-01 01:48:51.306] [ERROR] [2024-04-01 01:48:51.306] [ERROR] During handling of the above exception, another exception occurred: [2024-04-01 01:48:51.306] [ERROR] [2024-04-01 01:48:51.306] [ERROR] Traceback (most recent call last): [2024-04-01 01:48:51.306] [ERROR] File "paramiko/sftp_client.py", line 852, in _read_response [2024-04-01 01:48:51.306] [ERROR] File "paramiko/sftp.py", line 201, in _read_packet [2024-04-01 01:48:51.306] [ERROR] File "paramiko/sftp.py", line 188, in _read_all [2024-04-01 01:48:51.306] [ERROR] EOFError [2024-04-01 01:48:51.306] [ERROR] [2024-04-01 01:48:51.307] [ERROR] During handling of the above exception, another exception occurred: [2024-04-01 01:48:51.307] [ERROR] [2024-04-01 01:48:51.307] [ERROR] Traceback (most recent call last): [2024-04-01 01:48:51.307] [ERROR] File "core.py", line 2020, in start_cluster [2024-04-01 01:48:51.307] [ERROR] File "core.py", line 2113, in _start_cluster [2024-04-01 01:48:51.307] [ERROR] File "core.py", line 188, in call_plugin [2024-04-01 01:48:51.307] [ERROR] File "_plugin.py", line 347, in __call__ [2024-04-01 01:48:51.307] [ERROR] File "_plugin.py", line 305, in _new_func [2024-04-01 01:48:51.307] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 541, in start [2024-04-01 01:48:51.307] [ERROR] if not start_cluster(): [2024-04-01 01:48:51.307] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 350, in start_cluster [2024-04-01 01:48:51.307] [ERROR] _ocp_lib(client, home_path, soft_dir, stdio) [2024-04-01 01:48:51.307] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 241, in _ocp_lib [2024-04-01 01:48:51.308] [ERROR] client.put_file(rpm, os.path.join(home_path, 'ocp-server/lib/', name)) [2024-04-01 01:48:51.308] [ERROR] File "_plugin.py", line 225, in new_method [2024-04-01 01:48:51.308] [ERROR] File "_stdio.py", line 894, in func_wrapper [2024-04-01 01:48:51.308] [ERROR] File "ssh.py", line 530, in put_file [2024-04-01 01:48:51.308] [ERROR] File "_stdio.py", line 894, in func_wrapper [2024-04-01 01:48:51.308] [ERROR] File "ssh.py", line 558, in _client_put_file [2024-04-01 01:48:51.308] [ERROR] File "paramiko/sftp_client.py", line 759, in put [2024-04-01 01:48:51.308] [ERROR] File "paramiko/sftp_client.py", line 716, in putfo [2024-04-01 01:48:51.308] [ERROR] File "paramiko/util.py", line 304, in __exit__ [2024-04-01 01:48:51.308] [ERROR] File "paramiko/sftp_file.py", line 82, in close [2024-04-01 01:48:51.309] [ERROR] File "paramiko/sftp_file.py", line 104, in _close [2024-04-01 01:48:51.309] [ERROR] File "paramiko/sftp_client.py", line 822, in _request [2024-04-01 01:48:51.309] [ERROR] File "paramiko/sftp_client.py", line 854, in _read_response [2024-04-01 01:48:51.309] [ERROR] paramiko.ssh_exception.SSHException: Server connection dropped: [2024-04-01 01:48:51.309] [ERROR] [2024-04-01 01:48:51.309] [DEBUG] - sub start ref count to 0 [2024-04-01 01:48:51.309] [DEBUG] - export start [2024-04-01 01:48:51.309] [ERROR] ocp-server-ce start failed [2024-04-01 01:48:51.310] [DEBUG] - Call oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.2.2.1-101000012024030709.el7-3e30b9c0f183a5ef764ab754d5e517a671d96b38 [2024-04-01 01:48:51.310] [DEBUG] - import display [2024-04-01 01:48:51.313] [DEBUG] - add display ref count to 1 [2024-04-01 01:48:51.313] [DEBUG] -- execute sql: select * from oceanbase.__all_server. args: None [2024-04-01 01:48:51.317] [INFO] +----------------------------------------------+ [2024-04-01 01:48:51.318] [INFO] | observer | [2024-04-01 01:48:51.318] [INFO] +------------+---------+------+-------+--------+ [2024-04-01 01:48:51.318] [INFO] | ip | version | port | zone | status | [2024-04-01 01:48:51.318] [INFO] +------------+---------+------+-------+--------+ [2024-04-01 01:48:51.318] [INFO] | 165.3.87.9 | 4.2.2.1 | 2881 | zone1 | ACTIVE | [2024-04-01 01:48:51.318] [INFO] +------------+---------+------+-------+--------+ [2024-04-01 01:48:51.318] [INFO] obclient -h165.3.87.9 -P2881 -uroot -p'v/JgI&D95' -Doceanbase -A [2024-04-01 01:48:51.319] [DEBUG] - sub display ref count to 0 [2024-04-01 01:48:51.319] [DEBUG] - export display [2024-04-01 01:48:51.319] [DEBUG] - Call obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.2.3.0-3.el7-0490ebc04220def8d25cb9cac9ac61a4efa6d639 [2024-04-01 01:48:51.319] [DEBUG] - import display [2024-04-01 01:48:51.321] [DEBUG] - add display ref count to 1 [2024-04-01 01:48:51.321] [DEBUG] -- execute sql: show proxyconfig like "%port". args: None [2024-04-01 01:48:51.342] [INFO] +----------------------------------------------+ [2024-04-01 01:48:51.342] [INFO] | obproxy | [2024-04-01 01:48:51.342] [INFO] +------------+------+-----------------+--------+ [2024-04-01 01:48:51.342] [INFO] | ip | port | prometheus_port | status | [2024-04-01 01:48:51.342] [INFO] +------------+------+-----------------+--------+ [2024-04-01 01:48:51.342] [INFO] | 165.3.87.9 | 2883 | 2884 | active | [2024-04-01 01:48:51.342] [INFO] +------------+------+-----------------+--------+ [2024-04-01 01:48:51.344] [INFO] obclient -h165.3.87.9 -P2883 -uroot -p'v/JgI&D95' -Doceanbase -A [2024-04-01 01:48:51.344] [INFO] [2024-04-01 01:48:51.345] [DEBUG] - sub display ref count to 0 [2024-04-01 01:48:51.345] [DEBUG] - export display [2024-04-01 01:48:51.351] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes . [2024-04-01 01:48:51.351] [INFO] Trace ID: ed50ee88-efc9-11ee-8d2e-6645970001c2 [2024-04-01 01:48:51.351] [INFO] If you want to view detailed obd logs, please run: obd display-trace ed50ee88-efc9-11ee-8d2e-6645970001c2 [2024-04-01 01:48:51.414] [INFO] [ERROR] ocp-server-ce-py_script_start-4.2.1 RuntimeError: Server connection dropped: [2024-04-01 01:48:51.414] [INFO] [ERROR] ocp-server-ce start failed [2024-04-01 01:48:51.414] [INFO] +----------------------------------------------+ [2024-04-01 01:48:51.414] [INFO] | observer | [2024-04-01 01:48:51.414] [INFO] +------------+---------+------+-------+--------+ [2024-04-01 01:48:51.415] [INFO] | ip | version | port | zone | status | [2024-04-01 01:48:51.415] [INFO] +------------+---------+------+-------+--------+ [2024-04-01 01:48:51.415] [INFO] | 165.3.87.9 | 4.2.2.1 | 2881 | zone1 | ACTIVE | [2024-04-01 01:48:51.415] [INFO] +------------+---------+------+-------+--------+ [2024-04-01 01:48:51.415] [INFO] obclient -h165.3.87.9 -P2881 -uroot -p'v/JgI&D95' -Doceanbase -A [2024-04-01 01:48:51.415] [INFO] +----------------------------------------------+ [2024-04-01 01:48:51.415] [INFO] | obproxy | [2024-04-01 01:48:51.415] [INFO] +------------+------+-----------------+--------+ [2024-04-01 01:48:51.416] [INFO] | ip | port | prometheus_port | status | [2024-04-01 01:48:51.416] [INFO] +------------+------+-----------------+--------+ [2024-04-01 01:48:51.416] [INFO] | 165.3.87.9 | 2883 | 2884 | active | [2024-04-01 01:48:51.416] [INFO] +------------+------+-----------------+--------+ [2024-04-01 01:48:51.416] [INFO] obclient -h165.3.87.9 -P2883 -uroot -p'v/JgI&D95' -Doceanbase -A [2024-04-01 01:48:51.416] [INFO] [2024-04-01 01:48:51.416] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes . [2024-04-01 01:48:51.416] [INFO] Trace ID: ed50ee88-efc9-11ee-8d2e-6645970001c2 [2024-04-01 01:48:51.416] [INFO] If you want to view detailed obd logs, please run: obd display-trace ed50ee88-efc9-11ee-8d2e-6645970001c2 [2024-04-01 01:48:51.417] [INFO] [2024-04-01 01:48:51.418] [DEBUG] - unlock /root/.obd/lock/global [2024-04-01 01:48:51.418] [DEBUG] - unlock /root/.obd/lock/deploy_OCP [2024-04-01 01:48:51.419] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo