[INFO] Connect to observer
[2023-07-24 22:13:46.858] [DEBUG] – connect 172.17.90.244 -P2881 -uroot -pbiosan#17
[2023-07-24 22:13:47.065] [DEBUG] – connect 172.17.90.13 -P2881 -uroot -pbiosan#17
[2023-07-24 22:13:47.694] [DEBUG] – connect 172.17.90.15 -P2881 -uroot -pbiosan#17
[2023-07-24 22:13:51.336] [DEBUG] – connect 172.17.90.244 -P2881 -uroot -p
[2023-07-24 22:13:51.427] [DEBUG] - sub connect ref count to 0
[2023-07-24 22:13:51.427] [DEBUG] - export connect
[2023-07-24 22:13:51.428] [INFO] Initialize oceanbase-ce
[2023-07-24 22:13:51.429] [DEBUG] - Call oceanbase-ce-py_script_bootstrap-4.0.0.0 for oceanbase-ce-4.1.0.1-102000042023061314.el7-d03fafa6fa8ceb0636e4db05b5b5f6c3ac2256a3
[2023-07-24 22:13:51.429] [DEBUG] - import bootstrap
[2023-07-24 22:13:51.431] [DEBUG] - add bootstrap ref count to 1
[2023-07-24 22:13:51.431] [DEBUG] – execute sql: set session ob_query_timeout=1000000000
[2023-07-24 22:13:51.432] [DEBUG] – execute sql: set session ob_query_timeout=1000000000. args: None
[2023-07-24 22:13:51.435] [DEBUG] – execute sql: alter system bootstrap REGION “sys_region” ZONE “zone1” SERVER “172.17.90.244:2882”,REGION “sys_region” ZONE “zone2” SERVER “172.17.90.13:2882”,REGION “sys_region” ZONE “zone3” SERVER “172.17.90.15:2882”. args: None
[2023-07-24 22:14:27.564] [DEBUG] – OBD-5000: alter system bootstrap REGION “sys_region” ZONE “zone1” SERVER “172.17.90.244:2882”,REGION “sys_region” ZONE “zone2” SERVER “172.17.90.13:2882”,REGION “sys_region” ZONE “zone3” SERVER “172.17.90.15:2882” execute failed
[2023-07-24 22:14:27.568] [ERROR] Traceback (most recent call last):
[2023-07-24 22:14:27.568] [ERROR] File “core.py”, line 2328, in redeploy_cluster
[2023-07-24 22:14:27.568] [ERROR] File “core.py”, line 1676, in _start_cluster
[2023-07-24 22:14:27.568] [ERROR] File “core.py”, line 184, in call_plugin
[2023-07-24 22:14:27.568] [ERROR] File “_plugin.py”, line 323, in call
[2023-07-24 22:14:27.568] [ERROR] File “_plugin.py”, line 285, in _new_func
[2023-07-24 22:14:27.568] [ERROR] File “/home/admin/.obd/plugins/oceanbase-ce/4.0.0.0/bootstrap.py”, line 78, in bootstrap
[2023-07-24 22:14:27.568] [ERROR] raise_cursor.execute(sql, exc_level=‘verbose’)
[2023-07-24 22:14:27.568] [ERROR] File “_stdio.py”, line 885, in func_wrapper
[2023-07-24 22:14:27.568] [ERROR] File “/home/admin/.obd/plugins/oceanbase-ce/3.1.0/connect.py”, line 91, in execute
[2023-07-24 22:14:27.568] [ERROR] self.cursor.execute(sql, args)
[2023-07-24 22:14:27.569] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py”, line 148, in execute
[2023-07-24 22:14:27.569] [ERROR] result = self._query(query)
[2023-07-24 22:14:27.569] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/cursors.py”, line 310, in _query
[2023-07-24 22:14:27.569] [ERROR] conn.query(q)
[2023-07-24 22:14:27.569] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py”, line 548, in query
[2023-07-24 22:14:27.569] [ERROR] self._affected_rows = self._read_query_result(unbuffered=unbuffered)
[2023-07-24 22:14:27.569] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py”, line 775, in _read_query_result
[2023-07-24 22:14:27.569] [ERROR] result.read()
[2023-07-24 22:14:27.569] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py”, line 1156, in read
[2023-07-24 22:14:27.569] [ERROR] first_packet = self.connection._read_packet()
[2023-07-24 22:14:27.569] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/connections.py”, line 725, in _read_packet
[2023-07-24 22:14:27.569] [ERROR] packet.raise_for_error()
[2023-07-24 22:14:27.569] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/protocol.py”, line 221, in raise_for_error
[2023-07-24 22:14:27.569] [ERROR] err.raise_mysql_exception(self._data)
[2023-07-24 22:14:27.570] [ERROR] File “/home/admin/.oceanbase-all-in-one/obd/usr/obd/lib/site-packages/pymysql/err.py”, line 143, in raise_mysql_exception
[2023-07-24 22:14:27.570] [ERROR] raise errorclass(errno, errval)
[2023-07-24 22:14:27.570] [ERROR] pymysql.err.OperationalError: (4015, ‘System error’)
[2023-07-24 22:14:27.570] [ERROR]
[2023-07-24 22:14:27.570] [DEBUG] – execute sql: select column_value from oceanbase.__all_core_table where table_name = ‘__all_global_stat’ and column_name = ‘baseline_schema_version’. args: None
[2023-07-24 22:14:27.580] [DEBUG] - sub bootstrap ref count to 0
[2023-07-24 22:14:27.580] [DEBUG] - export bootstrap
[2023-07-24 22:14:27.603] [ERROR] Cluster init failed
[2023-07-24 22:14:27.610] [INFO] See https://www.oceanbase.com/product/ob-deployer/error-codes .
[2023-07-24 22:14:27.611] [INFO] Trace ID: 3c334cb2-2a2c-11ee-aa59-fa12f748ca00
[2023-07-24 22:14:27.611] [INFO] If you want to view detailed obd logs, please run: obd display-trace 3c334cb2-2a2c-11ee-aa59-fa12f748ca00
[2023-07-24 22:14:27.612] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 11
[2023-07-24 22:14:27.612] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 10
[2023-07-24 22:14:27.612] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 9
[2023-07-24 22:14:27.612] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 6
[2023-07-24 22:14:27.612] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 8
[2023-07-24 22:14:27.613] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 7
[2023-07-24 22:14:27.613] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 6
[2023-07-24 22:14:27.613] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 5
[2023-07-24 22:14:27.613] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 5
[2023-07-24 22:14:27.613] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 4
[2023-07-24 22:14:27.613] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 3
[2023-07-24 22:14:27.613] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 4
[2023-07-24 22:14:27.613] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 2
[2023-07-24 22:14:27.613] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 1
[2023-07-24 22:14:27.614] [DEBUG] - exclusive lock /home/admin/.obd/lock/mirror_and_repo release, count 0
[2023-07-24 22:14:27.614] [DEBUG] - try to get share lock /home/admin/.obd/lock/mirror_and_repo
[2023-07-24 22:14:27.614] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 3
[2023-07-24 22:14:27.614] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 2
[2023-07-24 22:14:27.614] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 1
[2023-07-24 22:14:27.614] [DEBUG] - share lock /home/admin/.obd/lock/mirror_and_repo release, count 0
[2023-07-24 22:14:27.614] [DEBUG] - unlock /home/admin/.obd/lock/mirror_and_repo
[2023-07-24 22:14:27.614] [DEBUG] - exclusive lock /home/admin/.obd/lock/deploy_acmeObd release, count 0
[2023-07-24 22:14:27.614] [DEBUG] - unlock /home/admin/.obd/lock/deploy_acmeObd
[2023-07-24 22:14:27.614] [DEBUG] - share lock /home/admin/.obd/lock/global release, count 0
[2023-07-24 22:14:27.615] [DEBUG] - unlock /home/admin/.obd/lock/global
看看日志，已经提示：obd display-trace 3c334cb2-2a2c-11ee-aa59-fa12f748ca00
执行一下，获取报错的具体日志，然后分析原因
多谢！
张老师，我这个日志就是这个trace打印出来的
看起来是bootstrap失败了，麻烦提供下observer.log日志吧，看看报了什么错
OBD-5000: alter system bootstrap REGION “sys_region” ZONE “zone1” SERVER “172.17.90.244:2882”,REGION “sys_region” ZONE “zone2” SERVER “172.17.90.13:2882”,REGION “sys_region” ZONE “zone3” SERVER “172.17.90.15:2882” execute failed
=== TENANTS MEMORY INFO ===
all_tenants_memstore_used= 2,097,152
[TENANT_MEMORY] tenant_id= 512 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 12,582,912 kv_cache_mem= 0
[TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,584,709,632 kv_cache_mem= 0
[TENANT_MEMORY] tenant_id= 506 mem_tenant_limit= 4,294,967,296 mem_tenant_hold= 29,360,128 kv_cache_mem= 0
[TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 33,554,432 kv_cache_mem= 0
[TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 20,971,520 kv_cache_mem= 0
[TENANT_MEMORY] tenant_id= 510 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 12,582,912 kv_cache_mem= 0
[TENANT_MEMORY] tenant_id= 1 active_memstore_used= 0 total_memstore_used= 2,097,152 total_memstore_hold= 2,097,152 memstore_freeze_trigger_limit= 214,748,360 memstore_limit= 1,073,741,800 mem_tenant_limit= 2,147,483,648 mem_tenant_hold= 421,138,432 kv_cache_mem= 0 max_mem_memstore_can_get_now= 1,728,442,368
[2023-07-25 15:31:30.806905] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:162) [4067][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=24] [CHUNK_MGR] free=34 pushes=3554 pops=3520 limit= 17,179,869,184 hold= 2,187,608,064 total_hold= 2,323,644,416 used= 2,112,802,816 freelist_hold= 74,805,248 large_freelist_hold= 3,502,080 maps= 637 unmaps= 58 large_maps= 32 large_unmaps= 0 huge_maps= 103 huge_unmaps= 57 memalign=0 resident_size= 3,027,849,216 virtual_memory_used= 3,660,218,368
[2023-07-25 15:31:30.806935] INFO dump (ob_concurrency_objpool.cpp:809) [4067][MemDumpTimer][T0][Y0-0000000000000000-0-0] [lt=13] [MEMORY] dump object freelist statistic:
[MEMORY] allocated | in-use | type size | cache type | free list name
[MEMORY] --------------------|--------------------|------------|------------|----------------------------------
[MEMORY] 0 | 0 | 144 | reclaim | ObThreadCache
[MEMORY] 0 | 0 | 256 | global | ObjFreeList
[2023-07-25 15:31:30.822894] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:389) [4269][TsMgr][T1][Y0-0000000000000000-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1690270290822885731, start_id=1690270290822885731, end_id=1690270290822885732)
[2023-07-25 15:31:30.852838] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [4557][T1_LSArchiveMgr][T1][YB42AC115A0D-0006014AB57CDEB4-0-0] [lt=25] gc stale ls task succ
[2023-07-25 15:31:30.853995] INFO [SERVER] check_config_mem_limit (ob_eliminate_task.cpp:78) [4550][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=67] change config mem limit(config_mem_limit_=67108864, mem_limit=64424509, tenant_id=1)
[2023-07-25 15:31:30.854025] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:213) [4550][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=28] sql audit evict task end(evict_high_mem_level=46137345, evict_high_size_level=9000000, evict_batch_count=0, elapse_time=0, size_used=5521, mem_used=16637952)
[2023-07-25 15:31:30.860396] INFO [SQL.RESV] check_table_exist_or_not (ob_dml_resolver.cpp:7554) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=12] table not exist(tenant_id=1, database_id=201001, table_name=__all_server, ret=-5019)
[2023-07-25 15:31:30.860430] WDIAG [SQL.RESV] resolve_table_relation_recursively (ob_dml_resolver.cpp:7512) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=33][errcode=-5019] synonym not exist(tenant_id=1, database_id=201001, table_name=__all_server, ret=-5019)
[2023-07-25 15:31:30.860444] WDIAG [SQL.RESV] resolve_table_relation_factor_normal (ob_dml_resolver.cpp:7349) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=14][errcode=-5019] fail to resolve table relation recursively(tenant_id=1, ret=-5019, database_id=201001, database_id=201001, table_name=__all_server, db_name=oceanbase)
[2023-07-25 15:31:30.860484] WDIAG [SQL.RESV] resolve_table_relation_factor (ob_dml_resolver.cpp:7194) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=39][errcode=-5019] resolve table relation factor failed(ret=-5019, table_name=__all_server)
[2023-07-25 15:31:30.860493] WDIAG [SQL.RESV] inner_resolve_sys_view (ob_dml_resolver.cpp:2581) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=7][errcode=-5019] fail to resolve table(ret=-5019)
[2023-07-25 15:31:30.860504] WDIAG [SQL.RESV] resolve_table_relation_factor_wrapper (ob_dml_resolver.cpp:2636) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=10][errcode=-5019] fail to resolve sys view(ret=-5019)
[2023-07-25 15:31:30.860512] WDIAG resolve_basic_table_without_cte (ob_dml_resolver.cpp:2732) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=4][errcode=-5019] Table 'oceanbase._all_server’ doesn’t exist
[2023-07-25 15:31:30.860523] WDIAG [SQL.RESV] resolve_basic_table_with_cte (ob_dml_resolver.cpp:13491) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=10][errcode=-5019] resolve base or alias table factor failed(ret=-5019)
[2023-07-25 15:31:30.860531] WDIAG [SQL.RESV] resolve_basic_table (ob_dml_resolver.cpp:13425) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=6][errcode=-5019] fail to resolve basic table with cte(ret=-5019)
[2023-07-25 15:31:30.860537] WDIAG [SQL.RESV] resolve_table (ob_dml_resolver.cpp:3144) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=5][errcode=-5019] resolve basic table failed(ret=-5019)
[2023-07-25 15:31:30.860542] WDIAG [SQL.RESV] resolve_table_list (ob_update_resolver.cpp:441) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=4][errcode=-5019] failed to resolve table(ret=-5019)
[2023-07-25 15:31:30.860546] WDIAG [SQL.RESV] resolve (ob_update_resolver.cpp:83) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=4][errcode=-5019] resolve table failed(ret=-5019)
[2023-07-25 15:31:30.860551] WDIAG [SQL.RESV] stmt_resolver_func (ob_resolver.cpp:155) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=3][errcode=-5019] execute stmt_resolver failed(ret=-5019, parse_tree.type=3074)
[2023-07-25 15:31:30.860561] WDIAG [SQL] generate_stmt (ob_sql.cpp:2664) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=5][errcode=-5019] failed to resolve(ret=-5019)
[2023-07-25 15:31:30.860567] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2786) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=4][errcode=-5019] Failed to generate stmt(ret=-5019, result.get_exec_context().need_disconnect()=false)
[2023-07-25 15:31:30.860574] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4458) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=4][errcode=-5019] Failed to generate plan(ret=-5019, result.get_exec_context().need_disconnect()=false)
[2023-07-25 15:31:30.860584] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=9][errcode=-5019] fail to handle physical plan(ret=-5019)
[2023-07-25 15:31:30.860604] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=18][errcode=-5019] fail to handle text query(stmt=UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882), ret=-5019)
[2023-07-25 15:31:30.860611] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:637) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=5][errcode=-5019] executor execute failed(ret=-5019)
[2023-07-25 15:31:30.860630] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:785) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=18][errcode=-5019] execute failed(ret=-5019, tenant_id=1, executor={ObIExecutor:, sql:"UPDATE all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882)"}, retry_cnt=0, local_sys_schema_version=1, local_tenant_schema_version=1)
[2023-07-25 15:31:30.860646] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=11][errcode=-5019] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err:-5019, err:“OB_TABLE_NOT_EXIST”, retry_type:0, client_ret:-5019}, need_retry=false)
[2023-07-25 15:31:30.860666] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=17][errcode=-5019] result set close failed(ret=-5019)
[2023-07-25 15:31:30.860678] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=10][errcode=-5019] result set close failed(ret=-5019)
[2023-07-25 15:31:30.860682] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:790) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=5][errcode=-5019] failed to close result(close_ret=-5019, ret=-5019)
[2023-07-25 15:31:30.860700] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:817) [4207][RSAsyncTask2][T1][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=6][errcode=-5019] failed to process record(executor={ObIExecutor:, sql:“UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882)”}, record_ret=-5019, ret=-5019)
[2023-07-25 15:31:30.860714] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:835) [4207][RSAsyncTask2][T0][YB42AC115A0D-0006014AB75CDECB-0-0] [lt=12][errcode=-5019] failed to process final(executor={ObIExecutor:, sql:“UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882)”}, aret=-5019, ret=-5019)
[2023-07-25 15:31:30.860728] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1823) [4207][RSAsyncTask2][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-5019] execute sql failed(ret=-5019, tenant_id=1, sql=UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882))
[2023-07-25 15:31:30.860736] INFO [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1912) [4207][RSAsyncTask2][T0][Y0-0000000000000000-0-0] [lt=7] execute write sql(ret=-5019, tenant_id=1, affected_rows=0, sql=UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882))
[2023-07-25 15:31:30.860750] WDIAG foreach_refactored (ob_hashtable.h:1308) [4207][RSAsyncTask2][T1][Y0-0000000000000000-0-0] [lt=6][errcode=0] hashtable not init
[2023-07-25 15:31:30.860761] WDIAG reset (ob_dependency_info.cpp:855) [4207][RSAsyncTask2][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4006] traversal ref obj version map failed(ret=-4006)
[2023-07-25 15:31:30.860778] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:889) [4207][RSAsyncTask2][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-5019] retry_while_no_tenant_resource failed(ret=-5019, tenant_id=1)
[2023-07-25 15:31:30.860789] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1791) [4207][RSAsyncTask2][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5019] execute_write failed(ret=-5019, tenant_id=1, sql=UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882), is_user_sql=false)
[2023-07-25 15:31:30.860796] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1780) [4207][RSAsyncTask2][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-5019] execute_write failed(ret=-5019, tenant_id=1, sql=“UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882)”)
[2023-07-25 15:31:30.860808] WDIAG [COMMON.MYSQLP] write (ob_mysql_proxy.cpp:151) [4207][RSAsyncTask2][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5019] execute sql failed(ret=-5019, conn=0x7f95ebfec050, start=1690270290859675, sql=UPDATE __all_server SET WITH_ROOTSERVER=0 WHERE WITH_ROOTSERVER=1 AND (SVR_IP, SVR_PORT) != (‘172.17.90.13’, 2882))
[2023-07-25 15:31:30.860878] WDIAG [SHARE] run2 (ob_async_task_queue.cpp:149) [4207][RSAsyncTask2][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-5019] task process failed, start retry(max retry time=9223372036854775801, retry interval=1000000, ret=-5019)
[2023-07-25 15:31:30.864807] INFO [SQL.RESV] check_table_exist_or_not (ob_dml_resolver.cpp:7554) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=7] table not exist(tenant_id=1, database_id=201001, table_name=__all_server, ret=-5019)
[2023-07-25 15:31:30.864834] WDIAG [SQL.RESV] resolve_table_relation_recursively (ob_dml_resolver.cpp:7512) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=25][errcode=-5019] synonym not exist(tenant_id=1, database_id=201001, table_name=__all_server, ret=-5019)
[2023-07-25 15:31:30.864845] WDIAG [SQL.RESV] resolve_table_relation_factor_normal (ob_dml_resolver.cpp:7349) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=11][errcode=-5019] fail to resolve table relation recursively(tenant_id=1, ret=-5019, database_id=201001, database_id=201001, table_name=__all_server, db_name=oceanbase)
[2023-07-25 15:31:30.864854] WDIAG [SQL.RESV] resolve_table_relation_factor (ob_dml_resolver.cpp:7194) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=8][errcode=-5019] resolve table relation factor failed(ret=-5019, table_name=__all_server)
[2023-07-25 15:31:30.864863] WDIAG [SQL.RESV] inner_resolve_sys_view (ob_dml_resolver.cpp:2581) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=7][errcode=-5019] fail to resolve table(ret=-5019)
[2023-07-25 15:31:30.864877] WDIAG [SQL.RESV] resolve_table_relation_factor_wrapper (ob_dml_resolver.cpp:2636) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=13][errcode=-5019] fail to resolve sys view(ret=-5019)
[2023-07-25 15:31:30.864886] WDIAG resolve_basic_table_without_cte (ob_dml_resolver.cpp:2732) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=6][errcode=-5019] Table 'oceanbase._all_server’ doesn’t exist
[2023-07-25 15:31:30.864895] WDIAG [SQL.RESV] resolve_basic_table (ob_dml_resolver.cpp:13428) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=6][errcode=-5019] fail to resolve basic table with cte(ret=-5019)
[2023-07-25 15:31:30.864909] WDIAG [SQL.RESV] resolve_insert_field (ob_insert_resolver.cpp:423) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=13][errcode=-5019] fail to exec resolve_basic_table(*table_node, table_item)(ret=-5019)
[2023-07-25 15:31:30.864915] WDIAG [SQL.RESV] resolve_insert_clause (ob_insert_resolver.cpp:163) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=6][errcode=-5019] failed to resolve insert filed(ret=-5019)
[2023-07-25 15:31:30.864921] WDIAG [SQL.RESV] resolve (ob_insert_resolver.cpp:95) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=5][errcode=-5019] resolve single table insert failed(ret=-5019)
[2023-07-25 15:31:30.864935] WDIAG [SQL.RESV] stmt_resolver_func (ob_resolver.cpp:155) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=13][errcode=-5019] execute stmt_resolver failed(ret=-5019, parse_tree.type=3068)
[2023-07-25 15:31:30.864947] WDIAG [SQL] generate_stmt (ob_sql.cpp:2664) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=7][errcode=-5019] failed to resolve(ret=-5019)
[2023-07-25 15:31:30.864962] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2786) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=13][errcode=-5019] Failed to generate stmt(ret=-5019, result.get_exec_context().need_disconnect()=false)
[2023-07-25 15:31:30.864978] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4458) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=14][errcode=-5019] Failed to generate plan(ret=-5019, result.get_exec_context().need_disconnect()=false)
[2023-07-25 15:31:30.864993] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=13][errcode=-5019] fail to handle physical plan(ret=-5019)
[2023-07-25 15:31:30.865002] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=7][errcode=-5019] fail to handle text query(stmt=INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584), ret=-5019)
[2023-07-25 15:31:30.865017] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:637) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=14][errcode=-5019] executor execute failed(ret=-5019)
[2023-07-25 15:31:30.865032] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:785) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=14][errcode=-5019] execute failed(ret=-5019, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584)"}, retry_cnt=0, local_sys_schema_version=1, local_tenant_schema_version=1)
[2023-07-25 15:31:30.865048] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=12][errcode=-5019] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err:-5019, err:“OB_TABLE_NOT_EXIST”, retry_type:0, client_ret:-5019}, need_retry=false)
[2023-07-25 15:31:30.865069] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=18][errcode=-5019] result set close failed(ret=-5019)
[2023-07-25 15:31:30.865083] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=13][errcode=-5019] result set close failed(ret=-5019)
[2023-07-25 15:31:30.865090] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:790) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=6][errcode=-5019] failed to close result(close_ret=-5019, ret=-5019)
[2023-07-25 15:31:30.865129] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:817) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=29][errcode=-5019] failed to process record(executor={ObIExecutor:, sql:“INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584)”}, record_ret=-5019, ret=-5019)
[2023-07-25 15:31:30.865142] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:835) [4205][RSAsyncTask0][T0][YB42AC115A0D-0006014AB7CCDEF9-0-0] [lt=11][errcode=-5019] failed to process final(executor={ObIExecutor:, sql:“INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584)”}, aret=-5019, ret=-5019)
[2023-07-25 15:31:30.865159] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1823) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-5019] execute sql failed(ret=-5019, tenant_id=1, sql=INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584))
[2023-07-25 15:31:30.865175] INFO [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1912) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=15] execute write sql(ret=-5019, tenant_id=1, affected_rows=0, sql=INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584))
[2023-07-25 15:31:30.865197] WDIAG foreach_refactored (ob_hashtable.h:1308) [4205][RSAsyncTask0][T1][Y0-0000000000000000-0-0] [lt=15][errcode=0] hashtable not init
[2023-07-25 15:31:30.865219] WDIAG reset (ob_dependency_info.cpp:855) [4205][RSAsyncTask0][T1][Y0-0000000000000000-0-0] [lt=21][errcode=-4006] traversal ref obj version map failed(ret=-4006)
[2023-07-25 15:31:30.865239] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:889) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-5019] retry_while_no_tenant_resource failed(ret=-5019, tenant_id=1)
[2023-07-25 15:31:30.865254] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1791) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-5019] execute_write failed(ret=-5019, tenant_id=1, sql=INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584), is_user_sql=false)
[2023-07-25 15:31:30.865269] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1780) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-5019] execute_write failed(ret=-5019, tenant_id=1, sql=“INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584)”)
[2023-07-25 15:31:30.865287] WDIAG [COMMON.MYSQLP] write (ob_mysql_proxy.cpp:151) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=16][errcode=-5019] execute sql failed(ret=-5019, conn=0x7f95c9164050, start=1690270290863858, sql=INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584))
[2023-07-25 15:31:30.865335] WDIAG [SHARE] exec_insert_update (ob_dml_sql_splicer.cpp:1283) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-5019] execute sql failed(ret=-5019, sql=INSERT INTO __all_server (svr_ip, svr_port, id, zone, inner_port, with_rootserver, block_migrate_in_time, start_service_time, status, build_version, stop_time, with_partition, gmt_modified) VALUES (‘172.17.90.15’, 2882, 3, ‘zone3’, 2881, 0, 0, 0, ‘ACTIVE’, ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, 0, 1, usec_to_time(1690270290366584)) ON DUPLICATE KEY UPDATE id = 3, zone = ‘zone3’, inner_port = 2881, with_rootserver = 0, block_migrate_in_time = 0, start_service_time = 0, status = ‘ACTIVE’, build_version = ‘4.1.0.1_102000042023061314-43bca414d5065272a730c92a645c3e25768c1d05(Jun 13 2023 14:26:23)’, stop_time = 0, with_partition = 1, gmt_modified = usec_to_time(1690270290366584))
[2023-07-25 15:31:30.865409] WDIAG [SHARE] run2 (ob_async_task_queue.cpp:149) [4205][RSAsyncTask0][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-5019] task process failed, start retry(max retry time=9223372036854775801, retry interval=1000000, ret=-5019)
[2023-07-25 15:31:30.866196] INFO [SQL.RESV] check_table_exist_or_not (ob_dml_resolver.cpp:7554) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=11] table not exist(tenant_id=1, database_id=201001, table_name=__all_server, ret=-5019)
[2023-07-25 15:31:30.866224] WDIAG [SQL.RESV] resolve_table_relation_recursively (ob_dml_resolver.cpp:7512) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=27][errcode=-5019] synonym not exist(tenant_id=1, database_id=201001, table_name=__all_server, ret=-5019)
[2023-07-25 15:31:30.866237] WDIAG [SQL.RESV] resolve_table_relation_factor_normal (ob_dml_resolver.cpp:7349) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=11][errcode=-5019] fail to resolve table relation recursively(tenant_id=1, ret=-5019, database_id=201001, database_id=201001, table_name=__all_server, db_name=oceanbase)
[2023-07-25 15:31:30.866246] WDIAG [SQL.RESV] resolve_table_relation_factor (ob_dml_resolver.cpp:7194) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=9][errcode=-5019] resolve table relation factor failed(ret=-5019, table_name=__all_server)
[2023-07-25 15:31:30.866257] WDIAG [SQL.RESV] inner_resolve_sys_view (ob_dml_resolver.cpp:2581) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=8][errcode=-5019] fail to resolve table(ret=-5019)
[2023-07-25 15:31:30.866272] WDIAG [SQL.RESV] resolve_table_relation_factor_wrapper (ob_dml_resolver.cpp:2636) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=14][errcode=-5019] fail to resolve sys view(ret=-5019)
[2023-07-25 15:31:30.866280] WDIAG resolve_basic_table_without_cte (ob_dml_resolver.cpp:2732) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=5][errcode=-5019] Table 'oceanbase.all_server’ doesn’t exist
[2023-07-25 15:31:30.866285] WDIAG [SQL.RESV] resolve_basic_table_with_cte (ob_dml_resolver.cpp:13491) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=5][errcode=-5019] resolve base or alias table factor failed(ret=-5019)
[2023-07-25 15:31:30.866291] WDIAG [SQL.RESV] resolve_basic_table (ob_dml_resolver.cpp:13425) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=5][errcode=-5019] fail to resolve basic table with cte(ret=-5019)
[2023-07-25 15:31:30.866298] WDIAG [SQL.RESV] resolve_table (ob_dml_resolver.cpp:3144) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=6][errcode=-5019] resolve basic table failed(ret=-5019)
[2023-07-25 15:31:30.866312] WDIAG [SQL.RESV] resolve_from_clause (ob_select_resolver.cpp:3446) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=12][errcode=-5019] fail to exec resolve_table(*table_node, table_item)(ret=-5019)
[2023-07-25 15:31:30.866319] WDIAG [SQL.RESV] resolve_normal_query (ob_select_resolver.cpp:1033) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=6][errcode=-5019] fail to exec resolve_from_clause(parse_tree.children[PARSE_SELECT_FROM])(ret=-5019)
[2023-07-25 15:31:30.866324] WDIAG [SQL.RESV] resolve (ob_select_resolver.cpp:1240) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=5][errcode=-5019] resolve normal query failed(ret=-5019)
[2023-07-25 15:31:30.866330] WDIAG [SQL.RESV] select_stmt_resolver_func (ob_resolver.cpp:170) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=4][errcode=-5019] execute stmt_resolver failed(ret=-5019, parse_tree.type=3073)
[2023-07-25 15:31:30.866348] WDIAG [SQL] generate_stmt (ob_sql.cpp:2664) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=13][errcode=-5019] failed to resolve(ret=-5019)
[2023-07-25 15:31:30.866363] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2786) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=13][errcode=-5019] Failed to generate stmt(ret=-5019, result.get_exec_context().need_disconnect()=false)
[2023-07-25 15:31:30.866378] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4458) [4205][RSAsyncTask0][T1][YB42AC115A0D-0006014AB7CCDEFB-0-0] [lt=14][errcode=-5019] Failed to generate plan(ret=-5019, result.get_exec_context().need_disconnect()=false)
user:
username: admin
# password: your password if need
key_file: /home/admin/.ssh/id_rsa
# port: your ssh port, default 22
# timeout: ssh connection timeout (second), default 30
oceanbase-ce:
servers:
- name: server1
# Please don't use hostname, only IP can be supported
ip: 172.17.90.244
- name: server2
ip: 172.17.90.13
- name: server3
ip: 172.17.90.15
global:
# Please set devname as the network adaptor's name whose ip is in the setting of severs.
# if set severs as "127.0.0.1", please set devname as "lo"
# if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
devname: eth0
# if current hardware's memory capacity is smaller than 50G, please use the setting of "mini-single-example.yaml" and do a small adjustment.
memory_limit: 16G # The maximum running memory for an observer
# The reserved system memory. system_memory is reserved for general tenants. The default value is 30G.
system_memory: 8G
datafile_size: 30G # Size of the data file.
log_disk_size: 20G # The size of disk space used by the clog files.
enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
enable_syslog_recycle: true # Enable auto system log recycling or not. The default value is false.
max_syslog_file_count: 4 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
# Cluster name for OceanBase Database. The default value is obcluster. When you deploy OceanBase Database and obproxy, this value must be the same as the cluster_name for obproxy.
appname: obcluster
#root_password: observer # root user password
# proxyro_password: # proxyro user pasword, consistent with obproxy's observer_sys_password, can be empty
# ocp_meta_db: ocp_express # The database name of ocp express meta
# ocp_meta_username: meta # The username of ocp express meta
# ocp_meta_password: '' # The password of ocp express meta
# ocp_agent_monitor_password: '' # The password for obagent monitor user
ocp_meta_tenant: # The config for ocp express meta tenant
tenant_name: ocp
max_cpu: 1
memory_size: 2G
log_disk_size: 7680M # The recommend value is (4608 + (expect node num + expect tenant num) * 512) M.
# In this example , support multiple ob process in single node, so different process use different ports.
# If deploy ob cluster in multiple nodes, the port and path setting can be same.
root_password: xx#17
ocp_agent_monitor_password: xx#17
proxyro_password: xx#17
server1:
mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
# The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
home_path: /home/admin/observer
# The directory for data storage. The default value is $home_path/store.
# data_dir: /data
# The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
# redo_dir: /redo
zone: zone1
server2:
mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
# The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
home_path: /home/admin/observer
# The directory for data storage. The default value is $home_path/store.
# data_dir: /data
# The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
# redo_dir: /redo
zone: zone2
server3:
mysql_port: 2881 # External port for OceanBase Database. The default value is 2881. DO NOT change this value after the cluster is started.
rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882. DO NOT change this value after the cluster is started.
# The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
home_path: /home/admin/observer
# The directory for data storage. The default value is $home_path/store.
# data_dir: /data
# The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
# redo_dir: /redo
zone: zone3
obproxy-ce:
# Set dependent components for the component.
# When the associated configurations are not done, OBD will automatically get the these configurations from the dependent components.
depends:
- oceanbase-ce
servers:
- 172.17.90.244
global:
listen_port: 2883 # External port. The default value is 2883.
prometheus_listen_port: 2884 # The Prometheus port. The default value is 2884.
home_path: /home/admin/obproxy
# oceanbase root server list
# format: ip:mysql_port;ip:mysql_port. When a depends exists, OBD gets this value from the oceanbase-ce of the depends.
# rs_list: 192.168.1.2:2881;192.168.1.3:2881;192.168.1.4:2881
enable_cluster_checkout: false
# observer cluster name, consistent with oceanbase-ce's appname. When a depends exists, OBD gets this value from the oceanbase-ce of the depends.
# cluster_name: obcluster
skip_proxy_sys_private_check: true
enable_strict_kernel_release: false
# obproxy_sys_password: # obproxy sys user password, can be empty. When a depends exists, OBD gets this value from the oceanbase-ce of the depends.
# observer_sys_password: # proxyro user pasword, consistent with oceanbase-ce's proxyro_password, can be empty. When a depends exists, OBD gets this value from the oceanbase-ce of the depends.
obproxy_sys_password: xx#17
obagent:
depends:
- oceanbase-ce
servers:
- name: server1
# Please don't use hostname, only IP can be supported
ip: 172.17.90.244
- name: server2
ip: 172.17.90.13
- name: server3
ip: 172.17.90.15
global:
home_path: /home/admin/obagent
http_basic_auth_password: xx#17
【SOP 系列 01】OBD 部署 OceanBase 集群 yaml 文件修改说明 - 社区问答- OceanBase社区-分布式数据库
obproxy-ce模块
observer_sys_password 看下有注释吗
完全替换成这个人的配置还是这个报错
现在是报什么错误呢，能否截图看下？
rootservice.log 日志看一下
三个节点都是空的
这个日志看起来不太对，rootservice.log不应该没有的。麻烦把 /home/admin/observer/log 文件夹打个包上传吧，所有的日志都在里面。另外确认下，初始化失败后，observer进程还在运行吗？
