[admin@kv01 ~]$ obd display-trace 41310f76-c1b0-11ee-bfea-bc305bf89398 [2024-02-02 17:48:44.720] [DEBUG] - mkdir /home/admin/.obd/lock/ [2024-02-02 17:48:44.721] [DEBUG] - unknown lock mode [2024-02-02 17:48:44.721] [DEBUG] - try to get share lock /home/admin/.obd/lock/global [2024-02-02 17:48:44.721] [DEBUG] - share lock `/home/admin/.obd/lock/global`, count 1 [2024-02-02 17:48:44.721] [DEBUG] - cmd: ['obcluster'] [2024-02-02 17:48:44.721] [DEBUG] - opts: {'servers': '192.168.10.170', 'components': 'oceanbase-ce', 'force_delete': None, 'strict_check': None, 'without_parameter': None} [2024-02-02 17:48:44.721] [DEBUG] - Get Deploy by name [2024-02-02 17:48:44.721] [DEBUG] - mkdir /home/admin/.obd/cluster/ [2024-02-02 17:48:44.722] [DEBUG] - mkdir /home/admin/.obd/config_parser/ [2024-02-02 17:48:44.722] [DEBUG] - try to get exclusive lock /home/admin/.obd/lock/deploy_obcluster [2024-02-02 17:48:44.722] [DEBUG] - exclusive lock `/home/admin/.obd/lock/deploy_obcluster`, count 1 [2024-02-02 17:48:44.728] [DEBUG] - Deploy status judge [2024-02-02 17:48:44.728] [INFO] Get local repositories [2024-02-02 17:48:44.729] [DEBUG] - mkdir /home/admin/.obd/repository [2024-02-02 17:48:44.730] [DEBUG] - Get local repository oceanbase-ce-4.1.0.0-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.730] [DEBUG] - Search repository oceanbase-ce version: 4.1.0.0, tag: d598937b1cfb1df85e2c2231acf024e4994db533, release: None, package_hash: None [2024-02-02 17:48:44.730] [DEBUG] - try to get share lock /home/admin/.obd/lock/mirror_and_repo [2024-02-02 17:48:44.730] [DEBUG] - share lock `/home/admin/.obd/lock/mirror_and_repo`, count 1 [2024-02-02 17:48:44.730] [DEBUG] - mkdir /home/admin/.obd/repository/oceanbase-ce [2024-02-02 17:48:44.732] [DEBUG] - Found repository oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.732] [DEBUG] - Get local repository obproxy-ce-4.1.0.0-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.733] [DEBUG] - Search repository obproxy-ce version: 4.1.0.0, tag: 2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe, release: None, package_hash: None [2024-02-02 17:48:44.733] [DEBUG] - share lock `/home/admin/.obd/lock/mirror_and_repo`, count 2 [2024-02-02 17:48:44.733] [DEBUG] - mkdir /home/admin/.obd/repository/obproxy-ce [2024-02-02 17:48:44.735] [DEBUG] - Found repository obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.860] [DEBUG] - Get deploy config [2024-02-02 17:48:44.904] [INFO] Search plugins [2024-02-02 17:48:44.905] [DEBUG] - Searching start_check plugin for components ... [2024-02-02 17:48:44.905] [DEBUG] - Searching start_check plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.905] [DEBUG] - mkdir /home/admin/.obd/plugins [2024-02-02 17:48:44.906] [DEBUG] - Found for oceanbase-ce-py_script_start_check-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:44.907] [DEBUG] - Searching start_check plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.907] [DEBUG] - Found for obproxy-ce-py_script_start_check-3.1.0 for obproxy-ce-4.1.0.0 [2024-02-02 17:48:44.907] [DEBUG] - Searching create_tenant plugin for components ... [2024-02-02 17:48:44.907] [DEBUG] - Searching create_tenant plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.907] [DEBUG] - Found for oceanbase-ce-py_script_create_tenant-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:44.907] [DEBUG] - Searching create_tenant plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.908] [DEBUG] - No such create_tenant plugin for obproxy-ce-4.1.0.0 [2024-02-02 17:48:44.908] [DEBUG] - Searching start plugin for components ... [2024-02-02 17:48:44.908] [DEBUG] - Searching start plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.908] [DEBUG] - Found for oceanbase-ce-py_script_start-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:44.908] [DEBUG] - Searching start plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.908] [DEBUG] - Found for obproxy-ce-py_script_start-3.1.0 for obproxy-ce-4.1.0.0 [2024-02-02 17:48:44.909] [DEBUG] - Searching connect plugin for components ... [2024-02-02 17:48:44.909] [DEBUG] - Searching connect plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.909] [DEBUG] - Found for oceanbase-ce-py_script_connect-3.1.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:44.909] [DEBUG] - Searching connect plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.909] [DEBUG] - Found for obproxy-ce-py_script_connect-3.1.0 for obproxy-ce-4.1.0.0 [2024-02-02 17:48:44.909] [DEBUG] - Searching bootstrap plugin for components ... [2024-02-02 17:48:44.909] [DEBUG] - Searching bootstrap plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.910] [DEBUG] - Found for oceanbase-ce-py_script_bootstrap-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:44.910] [DEBUG] - Searching bootstrap plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.910] [DEBUG] - Found for obproxy-ce-py_script_bootstrap-3.1.0 for obproxy-ce-4.1.0.0 [2024-02-02 17:48:44.910] [DEBUG] - Searching display plugin for components ... [2024-02-02 17:48:44.910] [DEBUG] - Searching display plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:44.911] [DEBUG] - Found for oceanbase-ce-py_script_display-3.1.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:44.911] [DEBUG] - Searching display plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:44.911] [DEBUG] - Found for obproxy-ce-py_script_display-3.1.0 for obproxy-ce-4.1.0.0 [2024-02-02 17:48:45.037] [INFO] Open ssh connection [2024-02-02 17:48:45.692] [INFO] Load cluster param plugin [2024-02-02 17:48:45.693] [DEBUG] - Searching param plugin for components ... [2024-02-02 17:48:45.693] [DEBUG] - Search param plugin for oceanbase-ce [2024-02-02 17:48:45.693] [DEBUG] - Found for oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:45.694] [DEBUG] - Applying oceanbase-ce-param-4.0.0.0 for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:46.517] [DEBUG] - Search param plugin for obproxy-ce [2024-02-02 17:48:46.518] [DEBUG] - Found for obproxy-ce-param-3.1.0 for obproxy-ce-4.1.0.0 [2024-02-02 17:48:46.518] [DEBUG] - Applying obproxy-ce-param-3.1.0 for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:46.719] [INFO] Cluster status check [2024-02-02 17:48:46.720] [DEBUG] - Searching status plugin for components ... [2024-02-02 17:48:46.720] [DEBUG] - Searching status plugin for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:46.721] [DEBUG] - Found for oceanbase-ce-py_script_status-3.1.0 for oceanbase-ce-4.1.0.0 [2024-02-02 17:48:46.722] [DEBUG] - Searching status plugin for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:46.722] [DEBUG] - Found for obproxy-ce-py_script_status-3.1.0 for obproxy-ce-4.1.0.0 [2024-02-02 17:48:46.722] [DEBUG] - Call oceanbase-ce-py_script_status-3.1.0 for oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 [2024-02-02 17:48:46.723] [DEBUG] - import status [2024-02-02 17:48:46.724] [DEBUG] - add status ref count to 1 [2024-02-02 17:48:46.725] [DEBUG] -- admin@192.168.10.155 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-02-02 17:48:46.757] [DEBUG] -- exited code 0 [2024-02-02 17:48:46.758] [DEBUG] -- admin@192.168.10.155 execute: ls /proc/250360 [2024-02-02 17:48:46.817] [DEBUG] -- exited code 0 [2024-02-02 17:48:46.818] [DEBUG] -- admin@192.168.10.158 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-02-02 17:48:46.839] [DEBUG] -- exited code 0 [2024-02-02 17:48:46.839] [DEBUG] -- admin@192.168.10.158 execute: ls /proc/170770 [2024-02-02 17:48:46.901] [DEBUG] -- exited code 0 [2024-02-02 17:48:46.902] [DEBUG] -- admin@192.168.10.170 execute: cat /home/admin/oceanbase-ce/run/observer.pid [2024-02-02 17:48:46.973] [DEBUG] -- exited code 0 [2024-02-02 17:48:46.974] [DEBUG] -- admin@192.168.10.170 execute: ls /proc/455040 [2024-02-02 17:48:47.086] [DEBUG] -- exited code 2, error output: [2024-02-02 17:48:47.087] [DEBUG] ls: cannot access /proc/455040: No such file or directory [2024-02-02 17:48:47.087] [DEBUG] [2024-02-02 17:48:47.087] [DEBUG] - sub status ref count to 0 [2024-02-02 17:48:47.087] [DEBUG] - export status [2024-02-02 17:48:47.087] [DEBUG] - oceanbase-ce-4.1.0.0-100000202023040520.el7-d598937b1cfb1df85e2c2231acf024e4994db533 cluster status is inconsistent [2024-02-02 17:48:47.088] [DEBUG] - Call obproxy-ce-py_script_status-3.1.0 for obproxy-ce-4.1.0.0-7.el7-2a9d9bf67f179dcca2a8c9e7c77373d94e7e2abe [2024-02-02 17:48:47.088] [DEBUG] - import status [2024-02-02 17:48:47.089] [DEBUG] - add status ref count to 1 [2024-02-02 17:48:47.089] [DEBUG] -- admin@192.168.10.155 execute: cat /home/admin/obproxy/run/obproxy-192.168.10.155-2883.pid [2024-02-02 17:48:47.107] [DEBUG] -- exited code 0 [2024-02-02 17:48:47.107] [DEBUG] -- admin@192.168.10.155 execute: ls /proc/251723 [2024-02-02 17:48:47.171] [DEBUG] -- exited code 0 [2024-02-02 17:48:47.172] [DEBUG] - sub status ref count to 0 [2024-02-02 17:48:47.172] [DEBUG] - export status [2024-02-02 17:48:47.242] [INFO] Deploy "obcluster" is running [2024-02-02 17:48:47.242] [INFO] Trace ID: 41310f76-c1b0-11ee-bfea-bc305bf89398 [2024-02-02 17:48:47.242] [INFO] If you want to view detailed obd logs, please run: obd display-trace 41310f76-c1b0-11ee-bfea-bc305bf89398 [2024-02-02 17:48:47.242] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 1 [2024-02-02 17:48:47.242] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 0 [2024-02-02 17:48:47.242] [DEBUG] - unlock /home/admin/.obd/lock/mirror_and_repo [2024-02-02 17:48:47.242] [DEBUG] - exclusive lock /home/admin/.obd/lock/deploy_obcluster release, count 0 [2024-02-02 17:48:47.243] [DEBUG] - unlock /home/admin/.obd/lock/deploy_obcluster [2024-02-02 17:48:47.243] [DEBUG] - share lock /home/admin/.obd/lock/global release, count 0 [2024-02-02 17:48:47.243] [DEBUG] - unlock /home/admin/.obd/lock/global