[2024-02-29 16:03:19.508] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - cmd: ['ocp01'] [2024-02-29 16:03:19.510] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - opts: {'servers': None, 'components': None} [2024-02-29 16:03:19.510] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - mkdir /root/.obd/lock/ [2024-02-29 16:03:19.511] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - unknown lock mode [2024-02-29 16:03:19.512] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/global [2024-02-29 16:03:19.513] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/global`, count 1 [2024-02-29 16:03:19.513] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Get Deploy by name [2024-02-29 16:03:19.513] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - mkdir /root/.obd/cluster/ [2024-02-29 16:03:19.514] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - mkdir /root/.obd/config_parser/ [2024-02-29 16:03:19.515] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - try to get exclusive lock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:03:19.515] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - exclusive lock `/root/.obd/lock/deploy_ocp01`, count 1 [2024-02-29 16:03:19.532] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Check the deploy status [2024-02-29 16:03:19.534] [00396958-d6d9-11ee-896f-005056915ea0] [INFO] Get local repositories [2024-02-29 16:03:19.536] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - mkdir /root/.obd/repository [2024-02-29 16:03:19.537] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Get local repository ocp-server-ce-4.2.1-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:03:19.538] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/mirror_and_repo [2024-02-29 16:03:19.539] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 1 [2024-02-29 16:03:19.669] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Get deploy config [2024-02-29 16:03:19.711] [00396958-d6d9-11ee-896f-005056915ea0] [INFO] Search plugins [2024-02-29 16:03:19.713] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Searching param plugin for components ... [2024-02-29 16:03:19.713] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Search param plugin for ocp-server-ce [2024-02-29 16:03:19.713] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - mkdir /root/.obd/plugins [2024-02-29 16:03:19.715] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Found for ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:03:19.715] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Applying ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:03:20.045] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Searching stop plugin for components ... [2024-02-29 16:03:20.046] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Searching stop plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:03:20.047] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_stop-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:03:20.114] [00396958-d6d9-11ee-896f-005056915ea0] [INFO] Open ssh connection [2024-02-29 16:03:20.117] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - host: 192.168.167.144, port: 22, user: admin, password: 111111 [2024-02-29 16:03:21.035] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Call ocp-server-ce-py_script_stop-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:03:21.036] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - import stop [2024-02-29 16:03:21.042] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - add stop ref count to 1 [2024-02-29 16:03:21.043] [00396958-d6d9-11ee-896f-005056915ea0] [INFO] Stop ocp-server [2024-02-29 16:03:21.046] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo cat /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:03:21.170] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:03:21.171] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/368246 [2024-02-29 16:03:21.363] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:03:21.365] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/368246/fd [2024-02-29 16:03:21.552] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:03:21.552] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- 192.168.167.144 ocp-server[pid: 368246] stopping... [2024-02-29 16:03:21.553] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo kill -9 368246 [2024-02-29 16:03:21.727] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:03:22.745] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - sub stop ref count to 0 [2024-02-29 16:03:22.745] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - export stop [2024-02-29 16:03:22.746] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - Set ocp01 deploy status to stopped [2024-02-29 16:03:22.746] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - dump deploy info to /root/.obd/cluster/ocp01/.data [2024-02-29 16:03:22.753] [00396958-d6d9-11ee-896f-005056915ea0] [INFO] ocp01 stopped [2024-02-29 16:03:22.763] [00396958-d6d9-11ee-896f-005056915ea0] [INFO] Trace ID: 00396958-d6d9-11ee-896f-005056915ea0 [2024-02-29 16:03:22.764] [00396958-d6d9-11ee-896f-005056915ea0] [INFO] If you want to view detailed obd logs, please run: obd display-trace 00396958-d6d9-11ee-896f-005056915ea0 [2024-02-29 16:03:22.764] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 0 [2024-02-29 16:03:22.764] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo [2024-02-29 16:03:22.765] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - exclusive lock /root/.obd/lock/deploy_ocp01 release, count 0 [2024-02-29 16:03:22.765] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - unlock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:03:22.765] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - share lock /root/.obd/lock/global release, count 0 [2024-02-29 16:03:22.765] [00396958-d6d9-11ee-896f-005056915ea0] [DEBUG] - unlock /root/.obd/lock/global [2024-02-29 16:04:28.944] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - cmd: ['ocp01'] [2024-02-29 16:04:28.944] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - opts: {'servers': None, 'components': None, 'force_delete': None, 'strict_check': None, 'without_parameter': None} [2024-02-29 16:04:28.944] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - mkdir /root/.obd/lock/ [2024-02-29 16:04:28.945] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - unknown lock mode [2024-02-29 16:04:28.945] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/global [2024-02-29 16:04:28.945] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/global`, count 1 [2024-02-29 16:04:28.946] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Get Deploy by name [2024-02-29 16:04:28.946] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - mkdir /root/.obd/cluster/ [2024-02-29 16:04:28.946] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - mkdir /root/.obd/config_parser/ [2024-02-29 16:04:28.947] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - try to get exclusive lock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:04:28.947] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - exclusive lock `/root/.obd/lock/deploy_ocp01`, count 1 [2024-02-29 16:04:28.955] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Deploy status judge [2024-02-29 16:04:28.956] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] Get local repositories [2024-02-29 16:04:28.957] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - mkdir /root/.obd/repository [2024-02-29 16:04:28.957] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Get local repository ocp-server-ce-4.2.1-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:28.958] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Search repository ocp-server-ce version: 4.2.1, tag: 58cf72891d75a2fa7c754bafc42d336525baf0b5, release: None, package_hash: None [2024-02-29 16:04:28.959] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/mirror_and_repo [2024-02-29 16:04:28.959] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 1 [2024-02-29 16:04:28.959] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - mkdir /root/.obd/repository/ocp-server-ce [2024-02-29 16:04:28.963] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Found repository ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:29.090] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Get deploy config [2024-02-29 16:04:29.113] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] Search plugins [2024-02-29 16:04:29.114] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching start_check plugin for components ... [2024-02-29 16:04:29.114] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching start_check plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:29.114] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - mkdir /root/.obd/plugins [2024-02-29 16:04:29.116] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:04:29.116] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching create_tenant plugin for components ... [2024-02-29 16:04:29.116] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching create_tenant plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:29.117] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - No such create_tenant plugin for ocp-server-ce-4.2.1 [2024-02-29 16:04:29.117] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching start plugin for components ... [2024-02-29 16:04:29.117] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching start plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:29.118] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:04:29.118] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching connect plugin for components ... [2024-02-29 16:04:29.118] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching connect plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:29.119] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_connect-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:04:29.119] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching bootstrap plugin for components ... [2024-02-29 16:04:29.119] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching bootstrap plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:29.120] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_bootstrap-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:04:29.120] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching display plugin for components ... [2024-02-29 16:04:29.120] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching display plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:29.121] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_display-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:04:29.251] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] Open ssh connection [2024-02-29 16:04:29.252] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - host: 192.168.167.144, port: 22, user: admin, password: 111111 [2024-02-29 16:04:29.778] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] Load cluster param plugin [2024-02-29 16:04:29.779] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Searching param plugin for components ... [2024-02-29 16:04:29.780] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Search param plugin for ocp-server-ce [2024-02-29 16:04:29.781] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Found for ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:04:29.781] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Applying ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:30.065] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - Call ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:04:30.065] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - import start_check [2024-02-29 16:04:30.077] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - add start_check ref count to 1 [2024-02-29 16:04:30.078] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] Check before start ocp-server [2024-02-29 16:04:30.084] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- metadb connect check [2024-02-29 16:04:30.085] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- connect 192.168.167.144 -P2883 -uroot@sys -p111111 [2024-02-29 16:04:30.093] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- execute sql: show DATABASES like "oceanbase". args: None [2024-02-29 16:04:30.099] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- time check [2024-02-29 16:04:30.099] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: date +"%Y-%m-%d %H:%M:%S" [2024-02-29 16:04:30.218] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:30.224] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- now: 2024-02-29 16:04:30 [2024-02-29 16:04:30.224] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- cursor: [2024-02-29 16:04:30.225] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- execute sql: SELECT NOW() now. args: None [2024-02-29 16:04:30.228] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- ob_time: 2024-02-29 16:04:30 [2024-02-29 16:04:30.228] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- execute sql: select * from oceanbase.DBA_OB_TENANTS where TENANT_NAME = %s. args: ['ocp_monitor'] [2024-02-29 16:04:31.370] [299c8488-d6d9-11ee-9fee-005056915ea0] [ERROR] OBD-4361: tenant(ocp_monitor) alread exist [2024-02-29 16:04:31.371] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- execute sql: select * from oceanbase.DBA_OB_TENANTS where TENANT_NAME = %s. args: ['ocp_meta'] [2024-02-29 16:04:31.624] [299c8488-d6d9-11ee-9fee-005056915ea0] [ERROR] OBD-4361: tenant(ocp_meta) alread exist [2024-02-29 16:04:31.624] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- oceanbase version check [2024-02-29 16:04:31.624] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo -n true [2024-02-29 16:04:31.773] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:31.774] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo cat /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:04:31.969] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:31.970] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/368246 [2024-02-29 16:04:32.164] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 2, error output: [2024-02-29 16:04:32.164] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] ls: cannot access '/proc/368246': No such file or directory [2024-02-29 16:04:32.164] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] [2024-02-29 16:04:32.165] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo id -u admin [2024-02-29 16:04:32.354] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:32.355] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{udp*,tcp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:04:32.531] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:32.533] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo java -version [2024-02-29 16:04:32.774] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:32.774] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- java version [2024-02-29 16:04:32.775] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- java_major_version 1.8.0 [2024-02-29 16:04:32.775] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- java_update_version 372 [2024-02-29 16:04:32.776] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: which clockdiff [2024-02-29 16:04:32.876] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:32.877] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: cat /proc/meminfo [2024-02-29 16:04:32.974] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:32.976] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: df --block-size=1024 [2024-02-29 16:04:33.084] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:33.085] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /dev, total: 33616629760 avail: 33616629760 [2024-02-29 16:04:33.085] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /dev/shm, total: 33637224448 avail: 33637224448 [2024-02-29 16:04:33.085] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /run, total: 33637224448 avail: 33619734528 [2024-02-29 16:04:33.086] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /sys/fs/cgroup, total: 33637224448 avail: 33637224448 [2024-02-29 16:04:33.086] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /, total: 83706777600 avail: 68395499520 [2024-02-29 16:04:33.086] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /boot, total: 2136997888 avail: 1897955328 [2024-02-29 16:04:33.086] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /redo, total: 85857402880 avail: 20799803392 [2024-02-29 16:04:33.086] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /opt, total: 53656686592 avail: 41060667392 [2024-02-29 16:04:33.087] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /data, total: 85857402880 avail: 9986277376 [2024-02-29 16:04:33.087] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /run/user/0, total: 6727442432 avail: 6727442432 [2024-02-29 16:04:33.087] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /run/user/1000, total: 6727442432 avail: 6727442432 [2024-02-29 16:04:33.087] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: df --block-size=1024 /home/admin/logs [2024-02-29 16:04:33.190] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:04:33.192] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] -- get disk info for path /, total: 83706777600 avail: 68395499520 [2024-02-29 16:04:33.230] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] [ERROR] OBD-4361: tenant(ocp_monitor) alread exist [2024-02-29 16:04:33.230] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] [ERROR] OBD-4361: tenant(ocp_meta) alread exist [2024-02-29 16:04:33.230] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] [2024-02-29 16:04:33.231] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - sub start_check ref count to 0 [2024-02-29 16:04:33.231] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - export start_check [2024-02-29 16:04:33.231] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - ocp-server-ce starting check failed. [2024-02-29 16:04:33.241] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes . [2024-02-29 16:04:33.241] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] Trace ID: 299c8488-d6d9-11ee-9fee-005056915ea0 [2024-02-29 16:04:33.241] [299c8488-d6d9-11ee-9fee-005056915ea0] [INFO] If you want to view detailed obd logs, please run: obd display-trace 299c8488-d6d9-11ee-9fee-005056915ea0 [2024-02-29 16:04:33.242] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 0 [2024-02-29 16:04:33.242] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo [2024-02-29 16:04:33.242] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - exclusive lock /root/.obd/lock/deploy_ocp01 release, count 0 [2024-02-29 16:04:33.242] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - unlock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:04:33.243] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - share lock /root/.obd/lock/global release, count 0 [2024-02-29 16:04:33.243] [299c8488-d6d9-11ee-9fee-005056915ea0] [DEBUG] - unlock /root/.obd/lock/global [2024-02-29 16:16:19.412] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - cmd: ['ocp01'] [2024-02-29 16:16:19.413] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - opts: {'servers': None, 'components': None, 'force_delete': None, 'strict_check': None, 'without_parameter': None} [2024-02-29 16:16:19.413] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - mkdir /root/.obd/lock/ [2024-02-29 16:16:19.413] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - unknown lock mode [2024-02-29 16:16:19.414] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/global [2024-02-29 16:16:19.414] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/global`, count 1 [2024-02-29 16:16:19.414] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Get Deploy by name [2024-02-29 16:16:19.414] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - mkdir /root/.obd/cluster/ [2024-02-29 16:16:19.414] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - mkdir /root/.obd/config_parser/ [2024-02-29 16:16:19.415] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - try to get exclusive lock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:16:19.415] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - exclusive lock `/root/.obd/lock/deploy_ocp01`, count 1 [2024-02-29 16:16:19.422] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Deploy status judge [2024-02-29 16:16:19.423] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] Get local repositories [2024-02-29 16:16:19.424] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - mkdir /root/.obd/repository [2024-02-29 16:16:19.424] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Get local repository ocp-server-ce-4.2.1-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.424] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Search repository ocp-server-ce version: 4.2.1, tag: 58cf72891d75a2fa7c754bafc42d336525baf0b5, release: None, package_hash: None [2024-02-29 16:16:19.425] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/mirror_and_repo [2024-02-29 16:16:19.425] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 1 [2024-02-29 16:16:19.425] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - mkdir /root/.obd/repository/ocp-server-ce [2024-02-29 16:16:19.428] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Found repository ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.556] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Get deploy config [2024-02-29 16:16:19.577] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] Search plugins [2024-02-29 16:16:19.578] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching start_check plugin for components ... [2024-02-29 16:16:19.578] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching start_check plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.578] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - mkdir /root/.obd/plugins [2024-02-29 16:16:19.580] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:16:19.580] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching create_tenant plugin for components ... [2024-02-29 16:16:19.580] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching create_tenant plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.581] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - No such create_tenant plugin for ocp-server-ce-4.2.1 [2024-02-29 16:16:19.581] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching start plugin for components ... [2024-02-29 16:16:19.581] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching start plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.581] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:16:19.581] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching connect plugin for components ... [2024-02-29 16:16:19.582] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching connect plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.582] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_connect-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:16:19.582] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching bootstrap plugin for components ... [2024-02-29 16:16:19.582] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching bootstrap plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.583] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_bootstrap-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:16:19.583] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching display plugin for components ... [2024-02-29 16:16:19.583] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching display plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:19.584] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_display-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:16:19.712] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] Open ssh connection [2024-02-29 16:16:19.715] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - host: 192.168.167.144, port: 22, user: admin, password: 111111 [2024-02-29 16:16:20.500] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] Load cluster param plugin [2024-02-29 16:16:20.502] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Searching param plugin for components ... [2024-02-29 16:16:20.502] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Search param plugin for ocp-server-ce [2024-02-29 16:16:20.504] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Found for ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:16:20.504] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Applying ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:20.932] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - Call ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:16:20.932] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - import start_check [2024-02-29 16:16:20.952] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - add start_check ref count to 1 [2024-02-29 16:16:20.953] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] Check before start ocp-server [2024-02-29 16:16:20.958] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- metadb connect check [2024-02-29 16:16:20.958] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- connect 192.168.167.144 -P2883 -uroot@sys -p111111 [2024-02-29 16:16:20.966] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- execute sql: show DATABASES like "oceanbase". args: None [2024-02-29 16:16:20.976] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- time check [2024-02-29 16:16:20.976] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: date +"%Y-%m-%d %H:%M:%S" [2024-02-29 16:16:21.046] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:21.051] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- now: 2024-02-29 16:16:21 [2024-02-29 16:16:21.051] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- cursor: [2024-02-29 16:16:21.051] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- execute sql: SELECT NOW() now. args: None [2024-02-29 16:16:21.053] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- ob_time: 2024-02-29 16:16:21 [2024-02-29 16:16:21.053] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- execute sql: select * from oceanbase.DBA_OB_TENANTS where TENANT_NAME = %s. args: ['ocp_monitor'] [2024-02-29 16:16:21.132] [d11570d4-d6da-11ee-a2dc-005056915ea0] [ERROR] OBD-4361: tenant(ocp_monitor) alread exist [2024-02-29 16:16:21.132] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- execute sql: select * from oceanbase.DBA_OB_TENANTS where TENANT_NAME = %s. args: ['ocp_meta'] [2024-02-29 16:16:21.185] [d11570d4-d6da-11ee-a2dc-005056915ea0] [ERROR] OBD-4361: tenant(ocp_meta) alread exist [2024-02-29 16:16:21.185] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- oceanbase version check [2024-02-29 16:16:21.186] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo -n true [2024-02-29 16:16:21.281] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:21.282] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo cat /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:16:21.412] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:21.413] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/368246 [2024-02-29 16:16:21.543] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 2, error output: [2024-02-29 16:16:21.543] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] ls: cannot access '/proc/368246': No such file or directory [2024-02-29 16:16:21.544] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] [2024-02-29 16:16:21.544] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo id -u admin [2024-02-29 16:16:21.692] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:21.693] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{udp*,tcp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:16:21.832] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:21.833] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo java -version [2024-02-29 16:16:22.095] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:22.096] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- java version [2024-02-29 16:16:22.096] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- java_major_version 1.8.0 [2024-02-29 16:16:22.097] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- java_update_version 372 [2024-02-29 16:16:22.097] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: which clockdiff [2024-02-29 16:16:22.205] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:22.206] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: cat /proc/meminfo [2024-02-29 16:16:22.308] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:22.309] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: df --block-size=1024 [2024-02-29 16:16:22.417] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:22.418] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /dev, total: 33616629760 avail: 33616629760 [2024-02-29 16:16:22.418] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /dev/shm, total: 33637224448 avail: 33637224448 [2024-02-29 16:16:22.419] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /run, total: 33637224448 avail: 33619734528 [2024-02-29 16:16:22.419] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /sys/fs/cgroup, total: 33637224448 avail: 33637224448 [2024-02-29 16:16:22.419] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /, total: 83706777600 avail: 68443901952 [2024-02-29 16:16:22.419] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /boot, total: 2136997888 avail: 1897955328 [2024-02-29 16:16:22.420] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /redo, total: 85857402880 avail: 20799803392 [2024-02-29 16:16:22.420] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /opt, total: 53656686592 avail: 41103233024 [2024-02-29 16:16:22.420] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /data, total: 85857402880 avail: 9986277376 [2024-02-29 16:16:22.420] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /run/user/0, total: 6727442432 avail: 6727442432 [2024-02-29 16:16:22.420] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /run/user/1000, total: 6727442432 avail: 6727442432 [2024-02-29 16:16:22.420] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: df --block-size=1024 /home/admin/logs [2024-02-29 16:16:22.520] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:16:22.521] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] -- get disk info for path /, total: 83706777600 avail: 68443897856 [2024-02-29 16:16:22.526] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] [ERROR] OBD-4361: tenant(ocp_monitor) alread exist [2024-02-29 16:16:22.526] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] [ERROR] OBD-4361: tenant(ocp_meta) alread exist [2024-02-29 16:16:22.526] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] [2024-02-29 16:16:22.526] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - sub start_check ref count to 0 [2024-02-29 16:16:22.526] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - export start_check [2024-02-29 16:16:22.526] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - ocp-server-ce starting check failed. [2024-02-29 16:16:22.534] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes . [2024-02-29 16:16:22.535] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] Trace ID: d11570d4-d6da-11ee-a2dc-005056915ea0 [2024-02-29 16:16:22.535] [d11570d4-d6da-11ee-a2dc-005056915ea0] [INFO] If you want to view detailed obd logs, please run: obd display-trace d11570d4-d6da-11ee-a2dc-005056915ea0 [2024-02-29 16:16:22.536] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 0 [2024-02-29 16:16:22.536] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo [2024-02-29 16:16:22.536] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - exclusive lock /root/.obd/lock/deploy_ocp01 release, count 0 [2024-02-29 16:16:22.536] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - unlock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:16:22.536] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - share lock /root/.obd/lock/global release, count 0 [2024-02-29 16:16:22.537] [d11570d4-d6da-11ee-a2dc-005056915ea0] [DEBUG] - unlock /root/.obd/lock/global [2024-02-29 16:22:14.433] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - cmd: ['ocp01'] [2024-02-29 16:22:14.433] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - opts: {'servers': None, 'components': None, 'force_delete': None, 'strict_check': None, 'without_parameter': None, 'skip_create_tenant': True} [2024-02-29 16:22:14.434] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - mkdir /root/.obd/lock/ [2024-02-29 16:22:14.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - unknown lock mode [2024-02-29 16:22:14.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/global [2024-02-29 16:22:14.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/global`, count 1 [2024-02-29 16:22:14.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Get Deploy by name [2024-02-29 16:22:14.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - mkdir /root/.obd/cluster/ [2024-02-29 16:22:14.438] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - mkdir /root/.obd/config_parser/ [2024-02-29 16:22:14.439] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - try to get exclusive lock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:22:14.439] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exclusive lock `/root/.obd/lock/deploy_ocp01`, count 1 [2024-02-29 16:22:14.456] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Deploy status judge [2024-02-29 16:22:14.457] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] Get local repositories [2024-02-29 16:22:14.459] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - mkdir /root/.obd/repository [2024-02-29 16:22:14.460] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Get local repository ocp-server-ce-4.2.1-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.460] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Search repository ocp-server-ce version: 4.2.1, tag: 58cf72891d75a2fa7c754bafc42d336525baf0b5, release: None, package_hash: None [2024-02-29 16:22:14.461] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - try to get share lock /root/.obd/lock/mirror_and_repo [2024-02-29 16:22:14.462] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - share lock `/root/.obd/lock/mirror_and_repo`, count 1 [2024-02-29 16:22:14.462] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - mkdir /root/.obd/repository/ocp-server-ce [2024-02-29 16:22:14.468] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Found repository ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.594] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Get deploy config [2024-02-29 16:22:14.639] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] Search plugins [2024-02-29 16:22:14.640] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching start_check plugin for components ... [2024-02-29 16:22:14.641] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching start_check plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.642] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - mkdir /root/.obd/plugins [2024-02-29 16:22:14.645] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:22:14.645] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching create_tenant plugin for components ... [2024-02-29 16:22:14.646] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching create_tenant plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.647] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - No such create_tenant plugin for ocp-server-ce-4.2.1 [2024-02-29 16:22:14.647] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching start plugin for components ... [2024-02-29 16:22:14.647] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching start plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.649] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:22:14.649] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching connect plugin for components ... [2024-02-29 16:22:14.649] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching connect plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.651] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_connect-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:22:14.651] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching bootstrap plugin for components ... [2024-02-29 16:22:14.651] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching bootstrap plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.653] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_bootstrap-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:22:14.653] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching display plugin for components ... [2024-02-29 16:22:14.653] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching display plugin for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:14.654] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Found for ocp-server-ce-py_script_display-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:22:14.776] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] Open ssh connection [2024-02-29 16:22:14.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - host: 192.168.167.144, port: 22, user: admin, password: 111111 [2024-02-29 16:22:15.697] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] Load cluster param plugin [2024-02-29 16:22:15.699] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Searching param plugin for components ... [2024-02-29 16:22:15.699] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Search param plugin for ocp-server-ce [2024-02-29 16:22:15.701] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Found for ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1 [2024-02-29 16:22:15.701] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Applying ocp-server-ce-param-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:16.135] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Call ocp-server-ce-py_script_start_check-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:16.136] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - import start_check [2024-02-29 16:22:16.156] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - add start_check ref count to 1 [2024-02-29 16:22:16.157] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] Check before start ocp-server [2024-02-29 16:22:16.162] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- metadb connect check [2024-02-29 16:22:16.163] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- connect 192.168.167.144 -P2883 -uroot@sys -p111111 [2024-02-29 16:22:16.172] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- execute sql: show DATABASES like "oceanbase". args: None [2024-02-29 16:22:16.218] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- time check [2024-02-29 16:22:16.219] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: date +"%Y-%m-%d %H:%M:%S" [2024-02-29 16:22:16.326] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:16.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- now: 2024-02-29 16:22:16 [2024-02-29 16:22:16.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- cursor: [2024-02-29 16:22:16.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- execute sql: SELECT NOW() now. args: None [2024-02-29 16:22:16.334] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- ob_time: 2024-02-29 16:22:16 [2024-02-29 16:22:16.334] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- oceanbase version check [2024-02-29 16:22:16.334] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo -n true [2024-02-29 16:22:16.507] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:16.508] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo cat /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:22:16.702] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:16.703] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/368246 [2024-02-29 16:22:16.930] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 2, error output: [2024-02-29 16:22:16.930] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] ls: cannot access '/proc/368246': No such file or directory [2024-02-29 16:22:16.931] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:22:16.932] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo id -u admin [2024-02-29 16:22:17.145] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:17.147] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{udp*,tcp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:22:17.306] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:17.307] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo java -version [2024-02-29 16:22:17.645] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:17.645] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- java version [2024-02-29 16:22:17.646] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- java_major_version 1.8.0 [2024-02-29 16:22:17.646] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- java_update_version 372 [2024-02-29 16:22:17.646] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: which clockdiff [2024-02-29 16:22:17.796] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:17.797] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: cat /proc/meminfo [2024-02-29 16:22:17.957] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:17.959] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: df --block-size=1024 [2024-02-29 16:22:18.111] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:18.113] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /dev, total: 33616629760 avail: 33616629760 [2024-02-29 16:22:18.113] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /dev/shm, total: 33637224448 avail: 33637224448 [2024-02-29 16:22:18.113] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /run, total: 33637224448 avail: 33619734528 [2024-02-29 16:22:18.113] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /sys/fs/cgroup, total: 33637224448 avail: 33637224448 [2024-02-29 16:22:18.113] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /, total: 83706777600 avail: 68443717632 [2024-02-29 16:22:18.114] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /boot, total: 2136997888 avail: 1897955328 [2024-02-29 16:22:18.114] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /redo, total: 85857402880 avail: 20799803392 [2024-02-29 16:22:18.114] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /opt, total: 53656686592 avail: 41105166336 [2024-02-29 16:22:18.114] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /data, total: 85857402880 avail: 9986277376 [2024-02-29 16:22:18.114] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /run/user/0, total: 6727442432 avail: 6727442432 [2024-02-29 16:22:18.114] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /run/user/1000, total: 6727442432 avail: 6727442432 [2024-02-29 16:22:18.114] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: df --block-size=1024 /home/admin/logs [2024-02-29 16:22:18.283] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:18.285] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- get disk info for path /, total: 83706777600 avail: 68443717632 [2024-02-29 16:22:18.390] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - sub start_check ref count to 0 [2024-02-29 16:22:18.390] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - export start_check [2024-02-29 16:22:18.391] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - Call ocp-server-ce-py_script_start-4.2.1 for ocp-server-ce-4.2.1-20231208144448.el7-58cf72891d75a2fa7c754bafc42d336525baf0b5 [2024-02-29 16:22:18.391] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - import start [2024-02-29 16:22:18.414] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - add start ref count to 1 [2024-02-29 16:22:18.414] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- metadb connect check [2024-02-29 16:22:18.416] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] Start ocp-server [2024-02-29 16:22:18.418] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- cp rpm & pos [2024-02-29 16:22:18.680] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - current remote_transporter RemoteTransporter.CLIENT [2024-02-29 16:22:18.681] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: mkdir -p /home/admin/ocp/ocp-server/lib && rm -fr /home/admin/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-02-29 16:22:18.811] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:18.812] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- send /root/.obd/mirror/local/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm to /home/admin/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-02-29 16:22:22.210] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - admin@192.168.167.144 execute: chmod 644 /home/admin/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-02-29 16:22:22.321] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exited code 0 [2024-02-29 16:22:22.323] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: mkdir -p /home/admin/software && rm -fr /home/admin/software/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-02-29 16:22:22.510] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:22.512] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- send /root/.obd/mirror/local/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm to /home/admin/software/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-02-29 16:22:25.849] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - admin@192.168.167.144 execute: chmod 644 /home/admin/software/ocp-agent-ce-4.2.1-20231208144448.el7.aarch64.rpm [2024-02-29 16:22:25.970] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exited code 0 [2024-02-29 16:22:25.971] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: mkdir -p /home/admin/ocp/ocp-server/lib && rm -fr /home/admin/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-02-29 16:22:26.160] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:26.161] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- send /root/.obd/mirror/local/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm to /home/admin/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-02-29 16:22:32.099] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - admin@192.168.167.144 execute: chmod 644 /home/admin/ocp/ocp-server/lib/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-02-29 16:22:32.211] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exited code 0 [2024-02-29 16:22:32.212] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: mkdir -p /home/admin/software && rm -fr /home/admin/software/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-02-29 16:22:32.440] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:32.441] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- send /root/.obd/mirror/local/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm to /home/admin/software/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-02-29 16:22:38.728] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - admin@192.168.167.144 execute: chmod 644 /home/admin/software/ocp-agent-ce-4.2.1-20231208144448.el7.x86_64.rpm [2024-02-29 16:22:38.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exited code 0 [2024-02-29 16:22:38.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: cat /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:22:39.012] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:39.014] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: ls /proc/368246 [2024-02-29 16:22:39.165] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 2, error output: [2024-02-29 16:22:39.166] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] ls: cannot access '/proc/368246': No such file or directory [2024-02-29 16:22:39.166] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:22:39.168] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- write cd /home/admin/ocp; export JDBC_URL=jdbc:oceanbase://192.168.167.144:2883/meta_database; export JDBC_USERNAME=root@ocp_meta;export JDBC_PASSWORD='OCPocp1234!@'; export JDBC_PUBLIC_KEY=;export OCP_INITIAL_ADMIN_PASSWORD='OCPocp1234!@';java -jar -Xms4g -Xmx4g -Docp.iam.encrypted-system-password=oceanbase /home/admin/ocp/lib/ocp-server.jar --bootstrap --progress-log=/home/admin/logs/bootstrap.log --with-property=server.port:8080 --with-property=logging.file.max-size:100MB --with-property=logging.file.total-size-cap:1GB --with-property=ocp.monitordb.host:192.168.167.144 --with-property=ocp.monitordb.username:root@ocp_monitor --with-property=ocp.monitordb.port:2883 --with-property=ocp.monitordb.password:'OCPocp1234!@' --with-property=ocp.monitordb.database:monitor_database --with-property=logging.file.name:/home/admin/logs/ocp-server.log --with-property=ocp.site.url:http://192.168.167.144:8080 --with-property=obsdk.ob.connection.mode:direct --with-property=ocp.file.local.built-in.dir:/home/admin/ocp/ocp-server/lib --with-property=ocp.log.download.tmp.dir:/home/admin/ocp/logs/ocp --with-property=ocp.file.local.dir:/home/admin/software > /dev/null 2>&1 & to admin@192.168.167.144:22: /home/admin/ocp/cmd.sh [2024-02-29 16:22:39.169] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: mkdir -p /home/admin/ocp && rm -fr /home/admin/ocp/cmd.sh [2024-02-29 16:22:39.326] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:22:39.327] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- send /tmp/tmp3t1sq_no to /home/admin/ocp/cmd.sh [2024-02-29 16:22:39.371] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - admin@192.168.167.144 execute: chmod 600 /home/admin/ocp/cmd.sh [2024-02-29 16:22:39.487] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exited code 0 [2024-02-29 16:22:39.488] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: chmod +x /home/admin/ocp/cmd.sh;sudo chown -R admin /home/admin/ocp/cmd.sh;sudo su - admin -c '/home/admin/ocp/cmd.sh' & [2024-02-29 16:22:39.865] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:23:39.925] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: ps -aux | grep -F 'java -jar -Xms4g -Xmx4g -Docp.iam.encrypted-system-password=oceanbase /home/admin/ocp/lib/ocp-server.jar --bootstrap' | grep -v grep | awk '{print $2}' [2024-02-29 16:23:40.086] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:23:40.088] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- write 2863060 to admin@192.168.167.144:22: /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:23:40.090] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: mkdir -p /home/admin/ocp/run && rm -fr /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:23:40.256] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:23:40.257] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- send /tmp/tmps8msrtl8 to /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:23:40.305] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - admin@192.168.167.144 execute: chmod 600 /home/admin/ocp/run/ocp-server.pid [2024-02-29 16:23:40.409] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exited code 0 [2024-02-29 16:23:40.530] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] ocp-server program health check [2024-02-29 16:23:40.531] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:23:40.532] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:23:40.645] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:23:40.646] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:23:40.815] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:23:40.816] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 39 [2024-02-29 16:23:55.820] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:23:55.821] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:23:56.016] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:23:56.017] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:23:56.239] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:23:56.240] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 38 [2024-02-29 16:24:11.255] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:24:11.256] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:24:11.344] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:11.345] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:24:11.471] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:11.472] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 37 [2024-02-29 16:24:26.491] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:24:26.492] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:24:26.695] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:26.695] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:24:26.837] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:26.837] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 36 [2024-02-29 16:24:41.853] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:24:41.853] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:24:41.955] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:41.955] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:24:42.082] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:42.082] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 35 [2024-02-29 16:24:57.097] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:24:57.097] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:24:57.190] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:57.190] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:24:57.314] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:24:57.315] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 34 [2024-02-29 16:25:12.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:25:12.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:25:12.485] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:25:12.487] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:25:12.674] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:25:12.675] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 33 [2024-02-29 16:25:27.690] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:25:27.691] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:25:27.812] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:25:27.813] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:25:27.975] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:25:27.975] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 32 [2024-02-29 16:25:42.991] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:25:42.991] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:25:43.157] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:25:43.157] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:25:43.350] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:25:43.350] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:25:43.351] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:25:43.351] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:26:13.400] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:26:13.400] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:26:13.400] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:26:13.400] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:26:13.400] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:26:13.400] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:26:13.401] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:26:13.402] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:26:13.403] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 31 [2024-02-29 16:26:28.418] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:26:28.419] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:26:28.589] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:26:28.590] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:26:28.736] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:26:28.737] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:26:28.737] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:26:28.738] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:26:58.783] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:26:58.784] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:26:58.785] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:26:58.786] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:26:58.786] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:26:58.786] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:26:58.786] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:26:58.786] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 30 [2024-02-29 16:27:13.800] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:27:13.801] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:27:13.892] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:27:13.892] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:27:14.020] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:27:14.020] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:27:14.021] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:27:14.021] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:27:44.067] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:27:44.067] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:27:44.067] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:27:44.067] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:27:44.067] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:27:44.067] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:27:44.067] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:27:44.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:27:44.069] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:27:44.070] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:27:44.070] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:27:44.070] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:27:44.070] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 29 [2024-02-29 16:27:59.085] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:27:59.086] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:27:59.262] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:27:59.263] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:27:59.444] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:27:59.445] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:27:59.445] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:27:59.445] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:28:29.493] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:28:29.493] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:28:29.493] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:28:29.493] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:28:29.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:28:29.495] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:28:29.496] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:28:29.496] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:28:29.496] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 28 [2024-02-29 16:28:44.511] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:28:44.512] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:28:44.667] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:28:44.668] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:28:44.875] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:28:44.876] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:28:44.876] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:28:44.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:29:14.926] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:29:14.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:29:14.928] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:29:14.929] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:29:14.930] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 27 [2024-02-29 16:29:29.945] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:29:29.946] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:29:30.223] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:29:30.223] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:29:30.425] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:29:30.427] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:29:30.427] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:29:30.428] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:30:00.476] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:30:00.476] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:30:00.476] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:30:00.476] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:30:00.476] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:30:00.476] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:30:00.477] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:30:00.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:30:00.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:30:00.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:30:00.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:30:00.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:30:00.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:30:00.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 26 [2024-02-29 16:30:15.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:30:15.494] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:30:15.628] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:30:15.630] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:30:15.795] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:30:15.796] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:30:15.796] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:30:15.797] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:30:45.845] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:30:45.845] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:30:45.845] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:30:45.846] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:30:45.847] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:30:45.848] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:30:45.848] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:30:45.848] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:30:45.848] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:30:45.848] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 25 [2024-02-29 16:31:00.864] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:31:00.864] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:31:00.955] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:31:00.956] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:31:01.085] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:31:01.086] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:31:01.086] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:31:01.087] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:31:31.136] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:31:31.136] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:31:31.136] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:31:31.137] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:31:31.137] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:31:31.137] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:31:31.137] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:31:31.138] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:31:31.138] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:31:31.138] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:31:31.138] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:31:31.139] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:31:31.139] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:31:31.139] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:31:31.139] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:31:31.139] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:31:31.140] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:31:31.140] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:31:31.140] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:31:31.140] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:31:31.141] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:31:31.141] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:31:31.141] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:31:31.141] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:31:31.141] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:31:31.142] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:31:31.142] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:31:31.142] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:31:31.142] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:31:31.143] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 24 [2024-02-29 16:31:46.146] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:31:46.147] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:31:46.311] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:31:46.313] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:31:46.487] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:31:46.490] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:31:46.490] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:31:46.491] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:32:16.541] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:32:16.541] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:32:16.542] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:32:16.542] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:32:16.542] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:32:16.543] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:32:16.543] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:32:16.543] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:32:16.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:32:16.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:32:16.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:32:16.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:32:16.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:32:16.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:32:16.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:32:16.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:32:16.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:32:16.547] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:32:16.547] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:32:16.547] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:32:16.547] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:32:16.547] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:32:16.548] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:32:16.548] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:32:16.548] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:32:16.548] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:32:16.548] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:32:16.548] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:32:16.549] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:32:16.550] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 23 [2024-02-29 16:32:31.564] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:32:31.565] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:32:31.720] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:32:31.721] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:32:31.858] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:32:31.859] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:32:31.859] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:32:31.859] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:33:01.906] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:33:01.906] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:33:01.906] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:33:01.906] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:33:01.906] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:33:01.907] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:33:01.907] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:33:01.907] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:33:01.907] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:33:01.907] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:33:01.907] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:33:01.907] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:33:01.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:33:01.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:33:01.910] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:33:01.910] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 22 [2024-02-29 16:33:16.917] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:33:16.917] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:33:17.043] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:33:17.043] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:33:17.189] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:33:17.190] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:33:17.190] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:33:17.190] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:33:47.236] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:33:47.237] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:33:47.238] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:33:47.239] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:33:47.239] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:33:47.239] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:33:47.239] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:33:47.239] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:33:47.239] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:33:47.240] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 21 [2024-02-29 16:34:02.255] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:34:02.255] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:34:02.355] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:34:02.356] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:34:02.538] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:34:02.540] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:34:02.540] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:34:02.540] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:34:32.590] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:34:32.590] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:34:32.590] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:34:32.590] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:34:32.590] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:34:32.591] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:34:32.592] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:34:32.593] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:34:32.593] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:34:32.593] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 20 [2024-02-29 16:34:47.608] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:34:47.608] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:34:47.760] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:34:47.763] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:34:47.944] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:34:47.945] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:34:47.946] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:34:47.947] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:35:17.996] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:35:17.996] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:35:17.996] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:35:17.996] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:35:17.997] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:35:17.998] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:35:17.998] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:35:17.998] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:35:17.998] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:35:17.998] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:35:17.998] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:35:17.999] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:35:18.000] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 19 [2024-02-29 16:35:33.015] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:35:33.016] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:35:33.167] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:35:33.168] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:35:33.385] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:35:33.386] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:35:33.386] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:35:33.387] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:36:03.434] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:36:03.435] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:36:03.436] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:36:03.437] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:36:03.438] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 18 [2024-02-29 16:36:18.449] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:36:18.449] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:36:18.659] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:36:18.660] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:36:18.828] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:36:18.828] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:36:18.828] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:36:18.829] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:36:48.876] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:36:48.876] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:36:48.876] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:36:48.876] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:36:48.876] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:36:48.877] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:36:48.878] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:36:48.879] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:36:48.879] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:36:48.879] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:36:48.879] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:36:48.879] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:36:48.879] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 17 [2024-02-29 16:37:03.894] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:37:03.895] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:37:04.045] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:37:04.046] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:37:04.201] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:37:04.202] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:37:04.202] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:37:04.202] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:37:34.252] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:37:34.252] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:37:34.253] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:37:34.253] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:37:34.253] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:37:34.253] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:37:34.254] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:37:34.254] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:37:34.254] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:37:34.254] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:37:34.255] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:37:34.255] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:37:34.255] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:37:34.255] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:37:34.256] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:37:34.256] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:37:34.256] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:37:34.256] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:37:34.256] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:37:34.257] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:37:34.257] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:37:34.257] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:37:34.257] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:37:34.257] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:37:34.258] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:37:34.258] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:37:34.258] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:37:34.259] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:37:34.259] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:37:34.260] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 16 [2024-02-29 16:37:49.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:37:49.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:37:49.378] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:37:49.380] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:37:49.521] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:37:49.523] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:37:49.524] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:37:49.524] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:38:19.575] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:38:19.575] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:38:19.576] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:38:19.576] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:38:19.576] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:38:19.576] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:38:19.577] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:38:19.577] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:38:19.577] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:38:19.577] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:38:19.577] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:38:19.578] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:38:19.578] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:38:19.578] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:38:19.578] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:38:19.578] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:38:19.579] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:38:19.579] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:38:19.579] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:38:19.579] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:38:19.579] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:38:19.579] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:38:19.580] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:38:19.580] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:38:19.580] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:38:19.580] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:38:19.580] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:38:19.580] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:38:19.581] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:38:19.581] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 15 [2024-02-29 16:38:34.588] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:38:34.589] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:38:34.710] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:38:34.712] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:38:34.906] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:38:34.908] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:38:34.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:38:34.909] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:39:04.959] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:39:04.960] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:39:04.961] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:39:04.962] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:39:04.962] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:39:04.962] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:39:04.962] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:39:04.962] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 14 [2024-02-29 16:39:19.974] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:39:19.975] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:39:20.121] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:39:20.123] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:39:20.282] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:39:20.282] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:39:20.283] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:39:20.283] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:39:50.329] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:39:50.329] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:39:50.329] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:39:50.329] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:39:50.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:39:50.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:39:50.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:39:50.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:39:50.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:39:50.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:39:50.330] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:39:50.331] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:39:50.332] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:39:50.332] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:39:50.332] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:39:50.332] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:39:50.332] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:39:50.332] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:39:50.332] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 13 [2024-02-29 16:40:05.347] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:40:05.348] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:40:05.519] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:40:05.520] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:40:05.726] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:40:05.727] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:40:05.727] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:40:05.728] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:40:35.778] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:40:35.779] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:40:35.780] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:40:35.781] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 12 [2024-02-29 16:40:50.795] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:40:50.796] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:40:50.923] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:40:50.925] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:40:51.136] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:40:51.137] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:40:51.137] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:40:51.138] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:41:21.193] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:41:21.194] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:41:21.194] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:41:21.195] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:41:21.195] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:41:21.195] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:41:21.196] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:41:21.196] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:41:21.196] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:41:21.196] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:41:21.197] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:41:21.197] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:41:21.197] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:41:21.197] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:41:21.198] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:41:21.198] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:41:21.198] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:41:21.199] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:41:21.199] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:41:21.199] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:41:21.200] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:41:21.200] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:41:21.200] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:41:21.200] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:41:21.201] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:41:21.201] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:41:21.201] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:41:21.201] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:41:21.202] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:41:21.203] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 11 [2024-02-29 16:41:36.216] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:41:36.217] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:41:36.341] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:41:36.343] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:41:36.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:41:36.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:41:36.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:41:36.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:42:06.580] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:42:06.581] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:42:06.581] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:42:06.581] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:42:06.581] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:42:06.582] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:42:06.582] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:42:06.582] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:42:06.582] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:42:06.582] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:42:06.583] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:42:06.583] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:42:06.583] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:42:06.584] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:42:06.584] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:42:06.584] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:42:06.584] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:42:06.585] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:42:06.585] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:42:06.585] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:42:06.585] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:42:06.586] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:42:06.586] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:42:06.586] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:42:06.586] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:42:06.586] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:42:06.587] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:42:06.587] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:42:06.587] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:42:06.588] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 10 [2024-02-29 16:42:21.603] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:42:21.604] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:42:21.709] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:42:21.711] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:42:21.860] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:42:21.862] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:42:21.862] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:42:21.863] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:42:51.913] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:42:51.913] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:42:51.914] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:42:51.914] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:42:51.914] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:42:51.914] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:42:51.915] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:42:51.915] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:42:51.915] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:42:51.915] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:42:51.916] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:42:51.916] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:42:51.916] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:42:51.916] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:42:51.916] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:42:51.916] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:42:51.917] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:42:51.917] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:42:51.917] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:42:51.917] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:42:51.917] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:42:51.918] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:42:51.918] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:42:51.918] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:42:51.918] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:42:51.919] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:42:51.919] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:42:51.919] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:42:51.919] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:42:51.920] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 9 [2024-02-29 16:43:06.921] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:43:06.922] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:43:07.068] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:43:07.070] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:43:07.218] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:43:07.219] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:43:07.219] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:43:07.219] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:43:37.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:43:37.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:43:37.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:43:37.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:43:37.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:43:37.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:43:37.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:43:37.267] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:43:37.268] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:43:37.268] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 8 [2024-02-29 16:43:52.283] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:43:52.284] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:43:52.371] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:43:52.372] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:43:52.497] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:43:52.497] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:43:52.498] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:43:52.498] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:44:22.544] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:44:22.545] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:44:22.546] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:44:22.547] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 7 [2024-02-29 16:44:37.562] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:44:37.562] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:44:37.709] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:44:37.710] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:44:37.925] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:44:37.926] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:44:37.926] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:44:37.927] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:45:07.976] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:45:07.977] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:45:07.977] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:45:07.978] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:45:07.978] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:45:07.978] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:45:07.979] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:45:07.979] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:45:07.979] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:45:07.979] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:45:07.980] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:45:07.980] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:45:07.980] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:45:07.980] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:45:07.980] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:45:07.981] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:45:07.981] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:45:07.981] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:45:07.981] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:45:07.982] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:45:07.982] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:45:07.982] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:45:07.982] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:45:07.983] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:45:07.983] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:45:07.983] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:45:07.984] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:45:07.984] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:45:07.984] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:45:07.986] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 6 [2024-02-29 16:45:23.001] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:45:23.001] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:45:23.091] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:45:23.092] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:45:23.246] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:45:23.247] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:45:23.248] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:45:23.248] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:45:53.294] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:45:53.295] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:45:53.296] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:45:53.297] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:45:53.297] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:45:53.297] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:45:53.297] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:45:53.297] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:45:53.297] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:45:53.297] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:45:53.298] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 5 [2024-02-29 16:46:08.313] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:46:08.313] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:46:08.468] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:46:08.468] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:46:08.661] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:46:08.662] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:46:08.662] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:46:08.663] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:46:38.719] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:46:38.719] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:46:38.719] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:46:38.719] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:46:38.720] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:46:38.720] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:46:38.720] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:46:38.720] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:46:38.720] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:46:38.721] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:46:38.721] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:46:38.721] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:46:38.721] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:46:38.721] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:46:38.721] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:46:38.722] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:46:38.722] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:46:38.722] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:46:38.722] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:46:38.723] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:46:38.723] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:46:38.723] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:46:38.723] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:46:38.723] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:46:38.723] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:46:38.724] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:46:38.724] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:46:38.724] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:46:38.724] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:46:38.725] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 4 [2024-02-29 16:46:53.727] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:46:53.727] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:46:53.904] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:46:53.906] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:46:54.107] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:46:54.108] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:46:54.108] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:46:54.109] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:47:24.161] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:47:24.161] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:47:24.161] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:47:24.161] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:47:24.162] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:47:24.162] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:47:24.162] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:47:24.162] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:47:24.162] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:47:24.162] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:47:24.163] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:47:24.163] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:47:24.163] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:47:24.163] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:47:24.163] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:47:24.163] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:47:24.164] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:47:24.164] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:47:24.164] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:47:24.164] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:47:24.164] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:47:24.165] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:47:24.165] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:47:24.165] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:47:24.165] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:47:24.166] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:47:24.166] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:47:24.166] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:47:24.166] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:47:24.167] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 3 [2024-02-29 16:47:39.182] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:47:39.183] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:47:39.313] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:47:39.314] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:47:39.478] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:47:39.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:47:39.479] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:47:39.480] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:48:09.526] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:48:09.526] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:48:09.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:48:09.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:48:09.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:48:09.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:48:09.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:48:09.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:48:09.527] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:48:09.528] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:48:09.529] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:48:09.530] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 2 [2024-02-29 16:48:24.538] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:48:24.538] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:48:24.665] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:48:24.666] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:48:24.837] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:48:24.838] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:48:24.838] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:48:24.838] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:48:54.887] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:48:54.888] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:48:54.889] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:48:54.890] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- failed to start 192.168.167.144 ocp-server, remaining retries: 1 [2024-02-29 16:49:09.890] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 192.168.167.144 program health check [2024-02-29 16:49:09.891] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: sudo ls /proc/2863060 [2024-02-29 16:49:10.042] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:49:10.045] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: bash -c 'cat /proc/net/{tcp*,udp*}' | awk -F' ' '{print $2,$10}' | grep '00000000:1F90' | awk -F' ' '{print $2}' | uniq [2024-02-29 16:49:10.214] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- exited code 0 [2024-02-29 16:49:10.214] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- 27208450 [2024-02-29 16:49:10.215] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] [2024-02-29 16:49:10.215] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] -- admin@192.168.167.144 execute: su - admin -c 'ls -l /proc/2863060/fd/ |grep -E "socket:\[(27208450)\]"' [2024-02-29 16:49:40.264] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:49:40.264] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 699, in recv [2024-02-29 16:49:40.264] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/buffered_pipe.py", line 164, in read [2024-02-29 16:49:40.264] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] paramiko.buffered_pipe.PipeTimeout [2024-02-29 16:49:40.264] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:49:40.264] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] During handling of the above exception, another exception occurred: [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] Traceback (most recent call last): [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1673, in start_cluster [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 1758, in _start_cluster [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "core.py", line 184, in call_plugin [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 324, in __call__ [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 286, in _new_func [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 704, in start [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] if not start_cluster(): [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 631, in start_cluster [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] confirm = confirm_port(client, pid, int(server_config["port"]), stdio, launch_user) [2024-02-29 16:49:40.265] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "/root/.obd/plugins/ocp-server-ce/4.2.1/start.py", line 145, in confirm_port [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ret = client.execute_command("""su - %s -c 'ls -l /proc/%s/fd/ |grep -E "socket:\[(%s)\]"'""" % (launch_user, pid, '|'.join(socket_inodes))) [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_plugin.py", line 223, in new_method [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 502, in execute_command [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "_stdio.py", line 888, in func_wrapper [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "ssh.py", line 463, in _execute_command [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/file.py", line 200, in read [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 1361, in _read [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] File "paramiko/channel.py", line 701, in recv [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] socket.timeout [2024-02-29 16:49:40.266] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] [2024-02-29 16:49:40.357] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] failed to start 192.168.167.144 ocp-server [2024-02-29 16:49:40.358] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] start ocp-server failed [2024-02-29 16:49:40.358] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - sub start ref count to 0 [2024-02-29 16:49:40.358] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - export start [2024-02-29 16:49:40.358] [a4b10782-d6db-11ee-ad27-005056915ea0] [ERROR] ocp-server-ce start failed [2024-02-29 16:49:40.371] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes . [2024-02-29 16:49:40.372] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] Trace ID: a4b10782-d6db-11ee-ad27-005056915ea0 [2024-02-29 16:49:40.373] [a4b10782-d6db-11ee-ad27-005056915ea0] [INFO] If you want to view detailed obd logs, please run: obd display-trace a4b10782-d6db-11ee-ad27-005056915ea0 [2024-02-29 16:49:40.374] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - share lock /root/.obd/lock/mirror_and_repo release, count 0 [2024-02-29 16:49:40.374] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - unlock /root/.obd/lock/mirror_and_repo [2024-02-29 16:49:40.375] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - exclusive lock /root/.obd/lock/deploy_ocp01 release, count 0 [2024-02-29 16:49:40.375] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - unlock /root/.obd/lock/deploy_ocp01 [2024-02-29 16:49:40.376] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - share lock /root/.obd/lock/global release, count 0 [2024-02-29 16:49:40.376] [a4b10782-d6db-11ee-ad27-005056915ea0] [DEBUG] - unlock /root/.obd/lock/global