[2023-07-23 11:04:38.646686] INFO  New syslog file info: [address: "192.168.1.1:1001", observer version: OceanBase_CE 4.1.0.0, revision: 100000202023040520-0765e69043c31bf86e83b5d618db0530cf31b707, sysname: Linux, os release: 3.10.0-1160.el7.x86_64, machine: x86_64, tz GMT offset: 08:00]
[2023-07-23 11:04:38.646488] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=403][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:38.646494] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:38.646502] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:38.646506] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:38.646510] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:38.646513] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:38.646519] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:38.646522] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:38.646525] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:38.646529] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:38.646532] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:38.646536] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:38.646556] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:57, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:38.646570] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196738}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478534081, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.646596] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196738}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478534081, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.646621] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081478635060, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196738}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478534081, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:38.646649] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:38.646655] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:38.646658] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:38.646664] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=94013472)
[2023-07-23 11:04:38.646677] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5] [packet retry query](TRACE=begin_ts=1690081478645755 2023-07-23 03:04:38.645755|[process_begin] u=0 in_queue_time:5985619, receive_ts:1690081472660135, enqueue_ts:1690081478645745|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=51 |[check_priv] u=8 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=1 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=2 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=230 |[sync_auto_value] u=431 |[close_plan_begin] u=52 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=78 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=27 |[process_end] u=1 run_ts:1690081478645757|total_timeu=921)
[2023-07-23 11:04:38.647502] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EB-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.648450] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.649359] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.649599] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1941-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.650237] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.655897] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:38.656344] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.656368] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=13194975, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.656378] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.662145] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:38.662153] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081478606075], now0=[mts=1690081478606075], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:38.662160] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908669}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478605617, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.662180] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=18][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:38.662190] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908669}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478605617, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.662206] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=15] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081478558206, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908669}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478605617, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:38.662232] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=26][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:38.662238] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:38.662241] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:38.662244] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:38.662249] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:38.662255] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=82, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:38.662261] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6] will sleep(sleep_us=82000, remain_us=22014277, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=82, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:38.666020] WDIAG [SHARE] refresh (ob_task_define.cpp:382) [3138][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=13][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4122, dropped:5411, tid:83032}])
[2023-07-23 11:04:38.666463] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=35][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.666475] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=13][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.666481] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1, svr="10.1.129.154:1001", timeout=9999965, timeout_timestamp=1690081488666420)
[2023-07-23 11:04:38.666492] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=10] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.666502] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:38.666508] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:38.666512] WDIAG [SQL] open (ob_result_set.cpp:157) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:38.666519] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:38.666526] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"})
[2023-07-23 11:04:38.666534] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] execute failed(ret=-4122, tenant_id=1, executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:38.666543] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:38.666558] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=12][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.666564] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.666567] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.666616] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.666623] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:38.666629] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1, sql=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:38.666635] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1)
[2023-07-23 11:04:38.666641] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1)
[2023-07-23 11:04:38.666647] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] query failed(ret=-4122, conn=0x7fd54a7f0050, start=1690081478666413, sql=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:38.666656] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:38.666664] WDIAG [SERVER] next_tenant_ (ob_iterate_private_virtual_table.cpp:255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] execute sql failed(ret=-4122, ret="OB_RPC_POST_ERROR", exec_tenant_id=1, cur_tenant_id=1, sql_=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:38.666672] WDIAG [SERVER] do_open (ob_iterate_private_virtual_table.cpp:138) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] switch to next tenant failed(ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:38.666678] WDIAG [SERVER] inner_get_next_row (ob_iterate_private_virtual_table.cpp:271) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] do open failed(ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:38.666681] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1285) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS](tenant_id=500, io_status=group_id: 0, group_name: OTHER_GROUPS, mode:  read, size: 2097152.00, iops:     2.00, rt:  3576.00)
[2023-07-23 11:04:38.666693] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1307) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS](tenant_id=500, ref_cnt=2, io_config={group_num_:0, memory_limit_:536870912, callback_thread_count_:8, unit_config_:{min_iops:10000, max_iops:50000, weight:10000}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=37693728, callback_queues=[0, 0, 0, 0, 0, 0, 0, 0])
[2023-07-23 11:04:38.666715] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666724] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666731] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666736] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666742] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666752] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666759] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666764] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666770] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666777] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666783] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666789] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666795] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666801] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666809] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666813] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:38.666684] WDIAG [COMMON] get_next_row (ob_virtual_table_iterator.cpp:372) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] fail to inner get next row(ret=-4122, scan_param_={tablet_id:{id:1}, ls_id:{id:1}, column_ids:[16, 20, 23, 27, 22, 26], index_id:12291, key_ranges:[{"range":"table_id:12291,group_idx:0,(MIN;MAX)"}], ss_key_ranges:[{"range":"table_id:12291,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1690081485923826, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup:0, reserved:0}, sql_mode:12582912, reserved_cell_count:6, schema_version:1686281724066464, query_begin_schema_version:1686562084609584, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[{type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:0, res_buf_off:416, dyn_buf_header_offset:400, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT UNSIGNED", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:16, res_buf_off:424, dyn_buf_header_offset:408, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484110}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT UNSIGNED", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:32, res_buf_off:432, dyn_buf_header_offset:416, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd5154841c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:48, res_buf_off:440, dyn_buf_header_offset:424, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484270}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:64, res_buf_off:448, dyn_buf_header_offset:432, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484320}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:80, res_buf_off:456, dyn_buf_header_offset:440, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd5154843d0}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7fd205206ce8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, table_param:{table_id:12291, output_projector:[0, 1, 2, 3, 4, 5], aggregate_projector:[], output_sel_mask:[true, true, true, true, true, true], pad_col_projector:[], main_read_info:{schema_column_count:13, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:13, is_oracle_mode:false, cols_index:[0, 6, 9, 13, 8, 12], memtable_cols_index:[0, 4, 7, 11, 6, 10], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:8, allocator:0x7fcfd79c2530, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:23, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:26, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fcfd79c2530, is_inited:true}, cols_param:null, index_read_info:NULL}}, use_lob_locator:false, rowid_version:0, rowid_projector:[], enable_lob_locator_v2:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0}, need_scn:false, need_switch_param:false})
[2023-07-23 11:04:38.666850] WDIAG [COMMON] get_next_row (ob_virtual_table_iterator.cpp:472) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=166][errcode=-4122] get next row failed(ret=-4122)
[2023-07-23 11:04:38.666855] WDIAG [SQL.ENG] get_next_row_with_das (ob_table_scan_op.cpp:1683) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] get next row from das result failed(ret=-4122)
[2023-07-23 11:04:38.666859] WDIAG [SQL.ENG] inner_get_next_row_for_tsc (ob_table_scan_op.cpp:1870) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] fail to get next row from ObNewRowIterator(ret=-4122)
[2023-07-23 11:04:38.666865] WDIAG [SQL.ENG] inner_get_next_row_implement (ob_table_scan_op.cpp:1847) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] failed to get next row(ret=-4122)
[2023-07-23 11:04:38.666868] WDIAG [SQL.ENG] inner_get_next_row (ob_table_scan_op.cpp:2827) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] get next row failed(ret=-4122)
[2023-07-23 11:04:38.666874] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] inner get next row failed(ret=-4122, type=3, op="PHY_TABLE_SCAN", op_id=0)
[2023-07-23 11:04:38.666881] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] get next row from operator failed(ret=-4122)
[2023-07-23 11:04:38.666885] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=3][errcode=-4122] get next row from exec result failed(ret=-4122)
[2023-07-23 11:04:38.666889] WDIAG [SERVER] response_query_result (ob_query_driver.cpp:233) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] fail to iterate and response(ret=-4122, row_num=0, can_retry=true)
[2023-07-23 11:04:38.666897] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:101) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] response query result fail(ret=-4122)
[2023-07-23 11:04:38.666902] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:29, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:2, client_ret:-4122}, need_retry=true)
[2023-07-23 11:04:38.666910] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:114) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] result response failed, check if need retry(ret=-4122, cli_ret=-4122, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:38.666944] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:121) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] close result set fail(cret=-4122)
[2023-07-23 11:04:38.666950] WDIAG [SERVER] do_process (obmp_query.cpp:846) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] execute query fail(ret=-4122, timeout_timestamp=1690081486023826)
[2023-07-23 11:04:38.666956] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] try to execute again(ret=-4122, type=1, retry_type=2, timeout_remain=7356871)
[2023-07-23 11:04:38.666970] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081478665912 2023-07-23 03:04:38.665912|[process_begin] u=0 in_queue_time:2642086, receive_ts:1690081476023826, enqueue_ts:1690081478665903|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=32 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=18 |[calc_partition_location_begin] u=448 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=51 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=8 |[do_open_plan_end] u=4 |[get_location_cache_begin] u=6 |[close_plan_begin] u=37 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=358 |[close_das_task_begin] u=1 |[close_das_task_end] u=25 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=25 |[process_end] u=1 run_ts:1690081478665915|total_timeu=1057)
[2023-07-23 11:04:38.686167] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=46][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.686177] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.686182] WDIAG [SQL.EXE] task_execute (ob_executor_rpc_impl.cpp:80) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1, svr="10.1.129.154:1001", timeout=6538546, timeout_timestamp=1690081485224690)
[2023-07-23 11:04:38.686192] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.686201] WDIAG [SQL.EXE] execute (ob_remote_task_executor.cpp:108) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4122] fail post task(ret=-4122)
[2023-07-23 11:04:38.686208] WDIAG [SQL.EXE] execute (ob_remote_job_executor.cpp:55) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4122] fail execute task(ret=-4122, *task_info={task_loc:{server:"10.1.129.154:1001", ob_task_id:{ob_job_id:{ob_execution_id:{server:"192.168.1.1:1001", execution_id:18446744073709551615, task_type:0, hash:4219408603045787357}, job_id:29239}, task_id:0, task_cnt:0}}, range_location:{part_locs:[], server:"0.0.0.0:0"}, location_idx:0, location_idx_list:[0], state:5, slice_count_pos:[], background:false, retry_times:0, location_idx_list:[0]})
[2023-07-23 11:04:38.686226] WDIAG [SQL.EXE] execute_with_plan (ob_remote_scheduler.cpp:124) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4122] fail execute remote job(ret=-4122)
[2023-07-23 11:04:38.686238] WDIAG [SQL.EXE] execute_with_plan (ob_remote_scheduler.cpp:141) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4122] fail to schedule, print job's status(ret=-4122, print_ret=0, job_status="{job_count:2, {task_count:0, state statistics:{}}, {task_count:1, task_info:{loc:"10.1.129.154:1001"ctrl:"192.168.1.1:1001"eid:18446744073709551615jid:29239tid:0pull_sid:0state:5}state statistics:{failed:1}}}")
[2023-07-23 11:04:38.686247] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:57) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4122] execute with plan failed(ret=-4122)
[2023-07-23 11:04:38.686253] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:38.686308] WDIAG [SQL] open (ob_result_set.cpp:157) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=54][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:38.686314] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:35, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:2, client_ret:-4122}, need_retry=true)
[2023-07-23 11:04:38.686323] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4122] result set open failed, check if need retry(ret=-4122, cli_ret=-4122, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:38.686334] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4122] close result set fail(cret=-4122)
[2023-07-23 11:04:38.686340] WDIAG [SERVER] do_process (obmp_query.cpp:846) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4122] execute query fail(ret=-4122, timeout_timestamp=1690081485224690)
[2023-07-23 11:04:38.686348] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4122] try to execute again(ret=-4122, type=1, retry_type=2, timeout_remain=6538343)
[2023-07-23 11:04:38.686363] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081478686091 2023-07-23 03:04:38.686091|[process_begin] u=0 in_queue_time:3461400, receive_ts:1690081475224690, enqueue_ts:1690081478686082|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=28 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=2 |[parse_job_end] u=4 |[remote_task_completed] u=68 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29239}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=47 |[do_open_plan_end] u=54 |[get_location_cache_begin] u=5 |[close_plan_begin] u=17 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=28 |[process_end] u=1 run_ts:1690081478686093|total_timeu=271)
[2023-07-23 11:04:38.691771] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.691788] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.691800] WDIAG [STORAGE.TRANS] post_cluster_heartbeat_rpc_ (ob_tenant_weak_read_service.cpp:803) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] post cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", cluster_service_master="10.1.129.154:1001", tenant_id_=1002, req={req_server:"192.168.1.1:1001", version:{val:1690081465077463222}, valid_part_count:4, total_part_count:4, generate_timestamp:1690081478691759}, cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.691827] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=26][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.691847] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:759) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4122] post cluster heartbeat rpc fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1002, local_server_version={val:1690081465077463222}, valid_part_count=4, total_part_count=4, generate_timestamp=1690081478691759)
[2023-07-23 11:04:38.691865] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:769) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4122] tenant weak read service do cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1002, last_post_cluster_heartbeat_tstamp_=1690081478591744, cluster_heartbeat_interval_=1000000, cluster_service_tablet_id={id:226}, cluster_service_master="10.1.129.154:1001")
[2023-07-23 11:04:38.691877] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.691893] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.691890] INFO  [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:808) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=false, can_update_version=false, cur_leader_epoch=0, start_service_tstamp_=0, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0)
[2023-07-23 11:04:38.691911] WDIAG [STORAGE.TRANS] post_cluster_heartbeat_rpc_ (ob_tenant_weak_read_service.cpp:803) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15][errcode=-4122] post cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", cluster_service_master="10.1.129.154:1001", tenant_id_=1, req={req_server:"192.168.1.1:1001", version:{val:1690081465406512168}, valid_part_count:1, total_part_count:1, generate_timestamp:1690081478691863}, cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.691938] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.691958] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:759) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4122] post cluster heartbeat rpc fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1, local_server_version={val:1690081465406512168}, valid_part_count=1, total_part_count=1, generate_timestamp=1690081478691863)
[2023-07-23 11:04:38.691974] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:769) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4122] tenant weak read service do cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1, last_post_cluster_heartbeat_tstamp_=1690081478591845, cluster_heartbeat_interval_=1000000, cluster_service_tablet_id={id:226}, cluster_service_master="10.1.129.154:1001")
[2023-07-23 11:04:38.691928] INFO  [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1029) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=21] get_min_unreplayed_log_info(lsn={lsn:8392989598}, scn={val:1690081465515197327}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1690081465607720, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:2108, ret_code_:0}}, next_to_submit_lsn_:{lsn:8392989598}, committed_end_lsn_:{lsn:8392989598}, next_to_submit_scn_:{val:1690081465515197327}, base_lsn_:{lsn:0}, base_scn_:{val:1}, iterator_:{iterator_impl:{buf_:0x7fd4e450d000, next_round_pread_size:2121728, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:8392989476}, end_lsn:{lsn:8392989598}, read_buf:{buf_len_:483328, buf_:0x7fd4e450d000}, block_size:67104768, log_storage_:0x7fd501ff8070}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1690081465515197326}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1690081465515197326}, data_checksum:2923900223, flag:1}}, init_mode_version:0, accumlate_checksum:383597832}}}})
[2023-07-23 11:04:38.692003] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=74] get wrs ts(ls_id={id:1}, delta=13176788, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.692015] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] get wrs ts(ls_id={id:1}, delta=13285473, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.692022] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.692034] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.692062] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=28] get wrs ts(ls_id={id:1001}, delta=13285235, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.692079] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.692107] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get wrs ts(ls_id={id:1002}, delta=13614522, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.692123] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.692150] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1003}, delta=13548663, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.692166] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.706405] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=47][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:38.706417] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=12][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039b45418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196739}, scn:1690081478635062}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196739}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478596177, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.706448] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=30][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196739}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478596177, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.706598] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=148][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:38.706604] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.706609] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:38.706617] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.706620] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:38.706626] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:38.706630] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:38.706636] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503967, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196739}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478596177, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039b4a628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196739}, scn:1690081478635062}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:38.707022] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=386][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:38.707027] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:38.707035] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:38.707039] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:38.707044] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:38.707048] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:38.707055] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:38.707059] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:38.707064] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:38.707068] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:38.707071] WDIAG [SQL] open (ob_result_set.cpp:159) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:38.707077] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:38.707098] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:52, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:38.707113] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196739}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478596177, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.707141] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196739}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478596177, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.707166] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=23] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081478635062, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196739}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478596177, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:38.707194] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:38.707201] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:38.707207] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:38.707215] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=94568024)
[2023-07-23 11:04:38.707230] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081478706298 2023-07-23 03:04:38.706298|[process_begin] u=0 in_queue_time:5431059, receive_ts:1690081473275239, enqueue_ts:1690081478706286|[start_sql] u=1 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=47 |[check_priv] u=13 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[insert_start] u=11 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=235 |[sync_auto_value] u=413 |[close_plan_begin] u=59 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=0 |[end_stmt] u=81 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=35 |[process_end] u=1 run_ts:1690081478706301|total_timeu=931)
[2023-07-23 11:04:38.707556] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=0][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.707568] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=12][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.707575] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999562, timeout_timestamp=1690081479707111)
[2023-07-23 11:04:38.707588] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=12] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.707597] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:38.707603] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:38.707609] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:38.707615] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:38.707621] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"})
[2023-07-23 11:04:38.707629] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:38.707638] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:38.707649] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.707654] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.707658] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.707667] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.707676] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:38.707684] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:38.707690] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:38.707697] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:38.707701] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] query failed(ret=-4122, conn=0x7fd54c1e8050, start=1690081478707503, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:38.707709] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:38.707713] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:38.707739] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:200282})
[2023-07-23 11:04:38.707747] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:200282}, add_timestamp:1690081478707082})
[2023-07-23 11:04:38.708168] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.708178] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.708187] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999592, timeout_timestamp=1690081479707755)
[2023-07-23 11:04:38.708196] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.708208] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=11][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:38.708214] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:38.708218] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:38.708224] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:38.708229] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"})
[2023-07-23 11:04:38.708236] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:38.708245] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:38.708256] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.708261] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.708264] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.708275] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.708282] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:38.708289] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:38.708300] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=11][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:38.708305] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:38.708311] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] query failed(ret=-4122, conn=0x7fd2f9d24050, start=1690081478708128, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:38.708318] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:38.708323] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:38.708346] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847555})
[2023-07-23 11:04:38.708353] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847555}, add_timestamp:1690081478707088})
[2023-07-23 11:04:38.708767] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.708776] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.708783] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999601, timeout_timestamp=1690081479708362)
[2023-07-23 11:04:38.708793] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.708801] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:38.708807] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:38.708813] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:38.708817] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:38.708823] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"})
[2023-07-23 11:04:38.708830] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:38.708839] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:38.708850] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.708855] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.708859] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.708866] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.708873] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:38.708879] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:38.708884] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:38.708888] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:38.708893] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] query failed(ret=-4122, conn=0x7fd5497e8050, start=1690081478708727, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:38.708900] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:38.708904] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:38.708926] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847591})
[2023-07-23 11:04:38.708939] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=12][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847591}, add_timestamp:1690081478707091})
[2023-07-23 11:04:38.709358] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.709367] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.709373] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999595, timeout_timestamp=1690081479708947)
[2023-07-23 11:04:38.709383] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.709391] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=8][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:38.709396] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:38.709402] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:38.709406] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:38.709412] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"})
[2023-07-23 11:04:38.709418] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:38.709428] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=10][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:38.709438] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=9][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.709444] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:38.709447] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.709456] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:38.709464] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:38.709471] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:38.709479] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:38.709483] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=4][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:38.709489] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=6][errcode=-4122] query failed(ret=-4122, conn=0x7fd41f004050, start=1690081478709318, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:38.709497] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:38.709503] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=5][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:38.709525] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847592})
[2023-07-23 11:04:38.709533] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847592}, add_timestamp:1690081478707093})
[2023-07-23 11:04:38.709541] WDIAG [SERVER] batch_process_tasks (ob_uniq_task_queue.h:498) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=7][errcode=-4122] fail to batch process task(ret=-4122)
[2023-07-23 11:04:38.709545] WDIAG [SERVER] run1 (ob_uniq_task_queue.h:449) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE71307931A-0-0] [lt=3][errcode=-4122] fail to batch execute task(ret=-4122, tasks.count()=4)
[2023-07-23 11:04:38.711314] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:118, local_retry_times:118, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:38.711336] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=19][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.711346] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.711354] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.711374] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.711442] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=13][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.711452] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.711464] WDIAG [STORAGE.TRANS] post (ob_gts_rpc.cpp:248) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", server="10.1.129.154:1001", msg={tenant_id:1001, srr:[mts=1690081478711437], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.711480] WDIAG [STORAGE.TRANS] query_gts_ (ob_gts_source.cpp:562) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=16][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", leader="10.1.129.154:1001", msg={tenant_id:1001, srr:[mts=1690081478711437], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.711493] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:38.711645] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:38.711657] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=13][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081572900748, now=[mts=1690081478655889], now0=[mts=1690081478655889], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:38.711670] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478540341, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:38.711704] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=29][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:1003}, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60})
[2023-07-23 11:04:38.711719] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=15][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=3222118872, *tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478540341, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:38.711747] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=27] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478540341, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:38.711792] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=43][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:501113, schema_version:1689916332789704, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:38.711803] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:38.711811] WDIAG [SQL] open (ob_result_set.cpp:157) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:38.711819] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=6][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:35, local_retry_times:0, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:2, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:38.711832] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] result set open failed, check if need retry(ret=-4283, cli_ret=-4283, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:38.711845] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] close result set fail(cret=-4283)
[2023-07-23 11:04:38.711853] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] execute query fail(ret=-4283, timeout_timestamp=1690081572970748)
[2023-07-23 11:04:38.711863] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] try to execute again(ret=-4283, type=1, retry_type=2, timeout_remain=94258886)
[2023-07-23 11:04:38.711880] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8] [packet retry query](TRACE=begin_ts=1690081478655847 2023-07-23 03:04:38.655847|[process_begin] u=0 in_queue_time:5685098, receive_ts:1690081472970748, enqueue_ts:1690081478655838|[start_sql] u=0 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=26 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=3 |[check_priv] u=2 |[plan_id] u=1 plan_id:45401|[do_open_plan_begin] u=1 plan_id:45401|[sql_start_stmt_begin] u=1 |[post_packet] u=8 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=55907 |[do_open_plan_end] u=8 |[close_plan_begin] u=30 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=33 |[process_end] u=1 run_ts:1690081478655849|total_timeu=56032)
[2023-07-23 11:04:38.712635] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3359][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.712817] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3365][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.713086] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3363][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.713153] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3361][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=23] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.713368] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3360][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.713490] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3358][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.713862] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3364][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.714040] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3362][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.714047] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3357][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.714481] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=1][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.714492] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.714503] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:38.714513] WDIAG [SQL.ENG] refresh_statistics (ob_px_tenant_target_monitor.cpp:114) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4122] query statistics failed(ret=-4122)
[2023-07-23 11:04:38.714523] WDIAG [COMMON] operator() (ob_px_target_mgr.h:143) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] target monitor refresh statistics failed(ret=-4122, px_tenant_info={v:1}, px_res_info->get_target_monitor()={is_init:true, tenant_id:1, server:"192.168.1.1:1001", dummy_cache_leader:"10.1.129.154:1001", role:2})
[2023-07-23 11:04:38.714546] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.714555] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.714563] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:38.714572] WDIAG [SQL.ENG] refresh_statistics (ob_px_tenant_target_monitor.cpp:114) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] query statistics failed(ret=-4122)
[2023-07-23 11:04:38.714580] WDIAG [COMMON] operator() (ob_px_target_mgr.h:143) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] target monitor refresh statistics failed(ret=-4122, px_tenant_info={v:1001}, px_res_info->get_target_monitor()={is_init:true, tenant_id:1001, server:"192.168.1.1:1001", dummy_cache_leader:"10.1.129.154:1001", role:2})
[2023-07-23 11:04:38.714815] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.714825] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.714832] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:38.714838] WDIAG [SQL.ENG] refresh_statistics (ob_px_tenant_target_monitor.cpp:114) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4122] query statistics failed(ret=-4122)
[2023-07-23 11:04:38.714846] WDIAG [COMMON] operator() (ob_px_target_mgr.h:143) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4122] target monitor refresh statistics failed(ret=-4122, px_tenant_info={v:1002}, px_res_info->get_target_monitor()={is_init:true, tenant_id:1002, server:"192.168.1.1:1001", dummy_cache_leader:"10.1.129.154:1001", role:2})
[2023-07-23 11:04:38.719634] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3369][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.720061] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3371][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.720229] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3366][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.720289] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3349][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=86/115, request doing=0/0)
[2023-07-23 11:04:38.720415] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3346][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=75/102, request doing=0/0)
[2023-07-23 11:04:38.720634] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3367][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=16] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.720662] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3352][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=80/105, request doing=0/0)
[2023-07-23 11:04:38.720709] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3345][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=87/100, request doing=0/0)
[2023-07-23 11:04:38.720788] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3353][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=89/110, request doing=0/0)
[2023-07-23 11:04:38.720857] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3372][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.721010] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3350][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=68/114, request doing=0/0)
[2023-07-23 11:04:38.721051] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3341][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=68/69, request doing=0/0)
[2023-07-23 11:04:38.721061] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3355][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.721067] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3374][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.721083] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3339][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=72/111, request doing=0/0)
[2023-07-23 11:04:38.721085] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3340][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=63/115, request doing=0/0)
[2023-07-23 11:04:38.721085] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3354][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=66/85, request doing=0/0)
[2023-07-23 11:04:38.721088] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3370][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.721088] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3351][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=73/90, request doing=0/0)
[2023-07-23 11:04:38.721261] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3368][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=14] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.721271] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3348][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=67/82, request doing=0/0)
[2023-07-23 11:04:38.721301] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3347][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=61/93, request doing=0/0)
[2023-07-23 11:04:38.721421] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3376][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:38.721526] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3344][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=87/104, request doing=0/0)
[2023-07-23 11:04:38.721672] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3343][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=80/96, request doing=0/0)
[2023-07-23 11:04:38.721694] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3342][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=79/92, request doing=0/0)
[2023-07-23 11:04:38.727202] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [3727][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=14] sql audit evict task end(evict_high_mem_level=412316860, evict_high_size_level=9000000, evict_batch_count=0, elapse_time=0, size_used=88081, mem_used=232931328)
[2023-07-23 11:04:38.729742] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:38.733331] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=12] Cache replace map node details(ret=0, replace_node_count=9, replace_time=12067, replace_start_pos=21800000, replace_num=100000)
[2023-07-23 11:04:38.736371] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.736380] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.736385] WDIAG [STORAGE.TRANS] post (ob_gts_rpc.cpp:248) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", server="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081478736367], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.736396] WDIAG [STORAGE.TRANS] query_gts_ (ob_gts_source.cpp:562) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", leader="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081478736367], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.736409] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081478736367]})
[2023-07-23 11:04:38.736418] WDIAG [STORAGE.TRANS] operator() (ob_ts_mgr.h:167) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] refresh gts failed(ret=-4122, ret="OB_RPC_POST_ERROR", gts_tenant_info={v:1})
[2023-07-23 11:04:38.736424] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.736430] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.736433] WDIAG [STORAGE.TRANS] post (ob_gts_rpc.cpp:248) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", server="10.1.129.154:1001", msg={tenant_id:1001, srr:[mts=1690081478736423], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.736438] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=15][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.736441] WDIAG [STORAGE.TRANS] query_gts_ (ob_gts_source.cpp:562) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", leader="10.1.129.154:1001", msg={tenant_id:1001, srr:[mts=1690081478736423], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.736444] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=7][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.736450] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=5] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081478736423]})
[2023-07-23 11:04:38.736449] WDIAG [RPC] call_rpc (ob_async_rpc_proxy.h:353) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=4][errcode=-4122] call rpc func failed(server="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3}, cluster_id=3, tenant_id=1, group_id=9, ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:38.736459] WDIAG [RPC] call (ob_async_rpc_proxy.h:287) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=10][errcode=-4122] call rpc func failed(server="10.1.129.154:1001", timeout=2000000, cluster_id=3, tenant_id=1, arg={addr:"10.1.129.154:1001", cluster_id:3}, group_id=9, ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:38.736469] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=9][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.736474] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.736484] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=8][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.736487] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=3][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.736491] WDIAG [RPC] call_rpc (ob_async_rpc_proxy.h:353) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=4][errcode=-4122] call rpc func failed(server="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3}, cluster_id=3, tenant_id=1, group_id=9, ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:38.736499] WDIAG [RPC] call (ob_async_rpc_proxy.h:287) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=8][errcode=-4122] call rpc func failed(server="10.1.129.155:1001", timeout=2000000, cluster_id=3, tenant_id=1, arg={addr:"10.1.129.155:1001", cluster_id:3}, group_id=9, ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:38.736511] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=11][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.736514] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.736584] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.736591] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.736595] WDIAG [STORAGE.TRANS] post (ob_gts_rpc.cpp:248) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", server="10.1.129.154:1001", msg={tenant_id:1002, srr:[mts=1690081478736582], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.736603] WDIAG [STORAGE.TRANS] query_gts_ (ob_gts_source.cpp:562) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", leader="10.1.129.154:1001", msg={tenant_id:1002, srr:[mts=1690081478736582], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:38.736602] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=31] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081478736597})
[2023-07-23 11:04:38.736615] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081478736582]})
[2023-07-23 11:04:38.736612] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC21-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:38.736623] INFO  [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:516) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, need_refresh=false, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081478736582]})
[2023-07-23 11:04:38.736628] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3291][][T0][YB420A01819C-0005FDE715FDDC21-0-0] [lt=13][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:38.736636] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3291][][T0][YB420A01819C-0005FDE715FDDC21-0-0] [lt=8][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:38.736656] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.736673] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=13][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.736692] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=9][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.736719] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=27][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.736744] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=24] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.736771] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=22] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:38.736791] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E3-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.736783] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04B-0-0] [lt=8] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081478736780, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:38.736803] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E3-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.737046] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=8][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.737057] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.737075] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BEC-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.737089] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BEC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.737140] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478736546 2023-07-23 03:04:38.736546|[process_begin] u=0 in_queue_time:2783225, receive_ts:1690081475953320, enqueue_ts:1690081478736538|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=39 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=19 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=6 |[do_open_plan_end] u=0 |[get_das_id_begin] u=42 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=12 |[calc_partition_location_begin] u=340 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=5 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=30 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=18 |[close_das_task_begin] u=0 |[close_das_task_end] u=19 |[start_end_stmt] u=17 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=12 |[process_end] u=0 run_ts:1690081478736548|total_timeu=593)
[2023-07-23 11:04:38.737207] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B06-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.737231] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B06-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.739521] INFO  [PALF] get_disk_usage (palf_env_impl.cpp:781) [3128][ServerGTimer][T1002][YB420A01819C-0005FDE72B4C6664-0-0] [lt=18] get_disk_usage(ret=0, capacity(MB):=60825, used(MB):=48597)
[2023-07-23 11:04:38.744322] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:82, local_retry_times:82, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:38.744336] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.744340] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.744345] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.744352] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.744391] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:38.750145] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1942-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.756402] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.756412] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.756438] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=13294925, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.756452] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.756865] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=29][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:38.756875] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=10][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196740}, scn:1690081478736178}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196740}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478646644, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.756905] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=29][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196740}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478646644, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.757061] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=155][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:38.757068] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.757073] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:38.757083] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=10][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.757087] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:38.757092] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:38.757098] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:38.757102] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503968, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196740}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478646644, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196740}, scn:1690081478736178}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:38.757524] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=421][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:38.757529] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:38.757537] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:38.757541] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:38.757545] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:38.757549] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:38.757555] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:38.757562] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:38.757566] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:38.757570] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:38.757576] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:38.757580] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:38.757604] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:58, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:38.757620] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196740}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478646644, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.757650] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=29] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196740}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478646644, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.757678] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081478736178, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196740}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478646644, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:38.757708] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:38.757716] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:38.757720] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:38.757727] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=93902408)
[2023-07-23 11:04:38.757743] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081478756748 2023-07-23 03:04:38.756748|[process_begin] u=0 in_queue_time:6096612, receive_ts:1690081472660135, enqueue_ts:1690081478756738|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=56 |[check_priv] u=9 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[insert_start] u=14 |[get_das_id_begin] u=14 |[get_das_id_end] u=1 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=241 |[sync_auto_value] u=448 |[close_plan_begin] u=65 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=87 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=34 |[process_end] u=1 run_ts:1690081478756750|total_timeu=994)
[2023-07-23 11:04:38.758050] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931B-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.758519] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.758962] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.759404] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.760336] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:167) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=6] sql audit evict mem start(evict_low_mem_level=28923920, evict_high_mem_level=49895441, size_used=43, mem_used=60312576)
[2023-07-23 11:04:38.760355] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:173) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=11] release old cannot free more memory
[2023-07-23 11:04:38.760361] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=6] sql audit evict task end(evict_high_mem_level=49895441, evict_high_size_level=90000, evict_batch_count=1, elapse_time=26, size_used=0, mem_used=60312576)
[2023-07-23 11:04:38.767221] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.767236] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:38.767242] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081478711436], now0=[mts=1690081478711436], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:38.767248] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:122, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.767276] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=28][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:38.767285] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:122, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.767305] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=19] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081478635064, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:122, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:38.767331] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=27][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:38.767340] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:38.767345] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:38.767348] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:38.767325] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478766810 2023-07-23 03:04:38.766810|[process_begin] u=0 in_queue_time:2742984, receive_ts:1690081476023826, enqueue_ts:1690081478766803|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=22 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=0 |[check_priv] u=2 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=1 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=2 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=13 |[calc_partition_location_begin] u=333 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=0 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=5 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=25 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=0 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=42 |[close_das_task_begin] u=1 |[close_das_task_end] u=33 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=6 |[process_end] u=1 run_ts:1690081478766812|total_timeu=514)
[2023-07-23 11:04:38.767352] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:38.767358] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=119, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:38.767365] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7] will sleep(sleep_us=100000, remain_us=16402482, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=119, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:38.777264] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [83138][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(evict_high_mem_level=510242114, evict_high_size_level=9000000, evict_batch_count=0, elapse_time=0, size_used=124218, mem_used=415948800)
[2023-07-23 11:04:38.787008] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=33][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478736780})
[2023-07-23 11:04:38.787024] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478736780}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:38.787038] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:38.787044] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:38.787048] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478736780}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:38.787068] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=20][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:38.787076] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:38.787079] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:38.787083] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:38.788300] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:38.788336] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478736780})
[2023-07-23 11:04:38.788351] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478736780}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:38.788365] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:38.788373] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:38.788377] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:38.788383] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:38.788386] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=2][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:38.788391] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:38.788395] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:38.788398] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:38.788403] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:38.788406] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:38.788411] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:38.788414] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:38.788420] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:38.788427] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:38.788432] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:38.788436] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:38.788443] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:36, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:38.788450] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:38.788457] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=6436233)
[2023-07-23 11:04:38.788476] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541947660181, cache_obj->added_lc()=false, cache_obj->get_object_id()=20494, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:38.788491] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081478786980 2023-07-23 03:04:38.786980|[process_begin] u=0 in_queue_time:3562290, receive_ts:1690081475224690, enqueue_ts:1690081478786973|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=3 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=20 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[pc_choose_plan] u=75 |[transform_with_outline_begin] u=50 |[transform_with_outline_end] u=1 |[resolve_begin] u=2 |[resolve_end] u=399 |[transform_begin] u=18 |[optimize_begin] u=618 |[tl_calc_part_id_end] u=115 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=157 |[query_end] u=48 |[process_end] u=1 run_ts:1690081478786984|total_timeu=1511)
[2023-07-23 11:04:38.790268] INFO  [STORAGE] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:1059) [3741][T1_MediumLoop][T1][Y0-0000000000000000-0-0] [lt=5] start schedule all tablet merge(merge_version=1690048801843563670)
[2023-07-23 11:04:38.791572] INFO  [STORAGE] schedule_all_tablets_medium (ob_tenant_tablet_scheduler.cpp:1161) [3741][T1_MediumLoop][T1][Y0-0000000000000000-0-0] [lt=8] finish schedule all tablet merge(merge_version=1690048801843563670, schedule_stats_={schedule_cnt:0, finish_cnt:794, wait_rs_validate_cnt:0}, tenant_merge_finish=true, merged_version_=1690048801843563670)
[2023-07-23 11:04:38.791584] INFO  [STORAGE] runTimerTask (ob_tenant_tablet_scheduler.cpp:136) [3741][T1_MediumLoop][T1][Y0-0000000000000000-0-0] [lt=11] MediumLoopTask(cost_ts=1319)
[2023-07-23 11:04:38.791846] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1}, delta=13275601, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.791870] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.791901] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get wrs ts(ls_id={id:1001}, delta=13385108, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.791918] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] get wrs ts(ls_id={id:1}, delta=13385346, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.791919] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.791934] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.791942] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1002}, delta=13714395, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.791955] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.791980] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1003}, delta=13648536, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.791996] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.800383] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:38.800394] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081478744385], now0=[mts=1690081478744385], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:38.800405] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908670}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478743554, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.800432] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=24][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:38.800445] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908670}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478743554, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.800468] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=22] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081478736177, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908670}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478743554, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:38.800502] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=34][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:38.800511] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:38.800518] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:38.800524] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:38.800530] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:38.800537] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=83, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:38.800546] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8] will sleep(sleep_us=83000, remain_us=21875991, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=83, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:38.817363] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=27][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:38.817374] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196741}, scn:1690081478736180}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196741}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478706668, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.817406] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=31][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196741}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478706668, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.817556] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=149][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:38.817563] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.817569] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:38.817577] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.817581] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:38.817586] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:38.817590] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:38.817594] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503969, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196741}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478706668, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196741}, scn:1690081478736180}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:38.817982] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=388][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:38.817988] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:38.817996] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:38.818000] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:38.818005] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:38.818009] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:38.818016] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:38.818021] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:38.818026] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:38.818030] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:38.818035] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:38.818039] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:38.818057] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:53, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:38.818073] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=10] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196741}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478706668, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.818100] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196741}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478706668, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.818125] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=23] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081478736180, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196741}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478706668, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:38.818154] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:38.818162] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:38.818168] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:38.818175] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=94457064)
[2023-07-23 11:04:38.818190] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081478817257 2023-07-23 03:04:38.817257|[process_begin] u=0 in_queue_time:5542017, receive_ts:1690081473275239, enqueue_ts:1690081478817248|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=53 |[check_priv] u=8 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=3 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=235 |[sync_auto_value] u=416 |[close_plan_begin] u=58 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=0 |[end_stmt] u=82 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=34 |[process_end] u=1 run_ts:1690081478817259|total_timeu=932)
[2023-07-23 11:04:38.818927] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EC-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.818956] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EC-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.819863] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.820758] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.821646] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.827396] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:38.829796] INFO  [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:118) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0}, this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"192.168.1.1:1001", switch_to_leader_ts:-1})
[2023-07-23 11:04:38.829808] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:38.836224] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [83154][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}])
[2023-07-23 11:04:38.836960] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081478836949]})
[2023-07-23 11:04:38.836973] INFO  [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] gts statistics(tenant_id=1, gts_rpc_cnt=0, get_gts_cache_cnt=51, get_gts_with_stc_cnt=4434, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=0, try_wait_gts_elapse_cnt=0)
[2023-07-23 11:04:38.836987] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081478836982]})
[2023-07-23 11:04:38.836995] INFO  [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:516) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1001, need_refresh=false, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081478836982]})
[2023-07-23 11:04:38.837005] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=18][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.837026] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.837040] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=12][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.837063] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.837075] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.837135] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081478837128]})
[2023-07-23 11:04:38.837277] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=16][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.837313] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=35][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.837346] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E4-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.837336] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=14][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.837381] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=40][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.837417] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=34] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.837456] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=34] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:38.837458] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=18] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081478837451})
[2023-07-23 11:04:38.837465] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC22-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:38.837473] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04C-0-0] [lt=13] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081478837469, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:38.837501] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C39-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.837874] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.837951] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478837405 2023-07-23 03:04:38.837405|[process_begin] u=0 in_queue_time:2884084, receive_ts:1690081475953320, enqueue_ts:1690081478837398|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=34 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=20 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=5 |[do_open_plan_end] u=0 |[get_das_id_begin] u=23 |[get_das_id_end] u=1 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=12 |[calc_partition_location_begin] u=330 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=5 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=24 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=17 |[close_das_task_begin] u=0 |[close_das_task_end] u=17 |[start_end_stmt] u=13 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[query_end] u=10 |[process_end] u=1 run_ts:1690081478837407|total_timeu=545)
[2023-07-23 11:04:38.843730] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73134578892, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:38.846257] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=32] Wash time detail, (compute_wash_size_time=154, refresh_score_time=1457, wash_time=1037)
[2023-07-23 11:04:38.850700] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1943-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.856463] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.856500] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13394776, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.856514] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.856820] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.867425] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:119, local_retry_times:119, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:38.867441] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=13][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.867447] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.867450] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.867460] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.867500] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.867507] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:38.867790] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:38.867801] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=12][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039b45418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196742}, scn:1690081478837155}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196742}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478757167, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.867833] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=31][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196742}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478757167, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.867980] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=147][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:38.867987] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.867992] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:38.868003] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=10][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.868007] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:38.868012] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:38.868016] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:38.868059] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.868149] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478867653 2023-07-23 03:04:38.867653|[process_begin] u=0 in_queue_time:2843827, receive_ts:1690081476023826, enqueue_ts:1690081478867646|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=22 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=2 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=1 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_das_id_begin] u=2 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=13 |[calc_partition_location_begin] u=329 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=5 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=24 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=45 |[close_das_task_begin] u=0 |[close_das_task_end] u=18 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=6 |[process_end] u=1 run_ts:1690081478867655|total_timeu=495)
[2023-07-23 11:04:38.868022] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503970, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196742}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478757167, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039b4a628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196742}, scn:1690081478837155}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:38.868414] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=391][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:38.868419] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:38.868426] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:38.868430] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:38.868434] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:38.868441] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:38.868446] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:38.868452] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:38.868456] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:38.868461] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:38.868465] WDIAG [SQL] open (ob_result_set.cpp:159) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:38.868471] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:38.868492] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:59, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:38.868506] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196742}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478757167, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.868535] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196742}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478757167, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.868560] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=23] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081478837155, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196742}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478757167, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:38.868589] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:38.868596] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:38.868600] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:38.868608] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=93791528)
[2023-07-23 11:04:38.868624] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081478867682 2023-07-23 03:04:38.867682|[process_begin] u=0 in_queue_time:6207546, receive_ts:1690081472660135, enqueue_ts:1690081478867673|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=55 |[check_priv] u=8 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=3 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=4 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=4 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=237 |[sync_auto_value] u=418 |[close_plan_begin] u=61 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=82 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=34 |[process_end] u=1 run_ts:1690081478867684|total_timeu=941)
[2023-07-23 11:04:38.869379] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE66087935F-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.870273] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE66087935F-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.871110] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE66087935F-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.871918] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE66087935F-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.872748] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83283][T1001_LogUpdate][T1001][Y0-0000000000000000-0-0] [lt=12][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.873113] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E6-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.875003] INFO  [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [3316][Blacklist][T0][Y0-0000000000000000-0-0] [lt=27] blacklist_loop exec finished(cost_time=46, is_enabled=true, send_cnt=0)
[2023-07-23 11:04:38.882475] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=36197, clean_start_pos=7800000, clean_num=200000)
[2023-07-23 11:04:38.883211] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:38.883225] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=13][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081572900748, now=[mts=1690081478827387], now0=[mts=1690081478827387], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:38.883239] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=14][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478710883, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:38.883279] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=35][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:1003}, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60})
[2023-07-23 11:04:38.883306] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=27][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=3222118872, *tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478710883, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:38.883337] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=29] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478710883, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:38.883392] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=53][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:501113, schema_version:1689916332789704, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:38.883406] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=13][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:38.883415] WDIAG [SQL] open (ob_result_set.cpp:157) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:38.883425] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:36, local_retry_times:0, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:2, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:38.883442] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=17][errcode=-4283] result set open failed, check if need retry(ret=-4283, cli_ret=-4283, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:38.883458] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] close result set fail(cret=-4283)
[2023-07-23 11:04:38.883467] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] execute query fail(ret=-4283, timeout_timestamp=1690081572970748)
[2023-07-23 11:04:38.883481] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4283] try to execute again(ret=-4283, type=1, retry_type=2, timeout_remain=94087268)
[2023-07-23 11:04:38.883503] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=10] [packet retry query](TRACE=begin_ts=1690081478827344 2023-07-23 03:04:38.827344|[process_begin] u=0 in_queue_time:5856596, receive_ts:1690081472970748, enqueue_ts:1690081478827336|[start_sql] u=1 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=27 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[pc_choose_plan] u=3 |[check_priv] u=3 |[plan_id] u=0 plan_id:45401|[do_open_plan_begin] u=2 plan_id:45401|[sql_start_stmt_begin] u=0 |[post_packet] u=9 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=56010 |[do_open_plan_end] u=10 |[close_plan_begin] u=37 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=4 |[query_end] u=43 |[process_end] u=2 run_ts:1690081478827346|total_timeu=56158)
[2023-07-23 11:04:38.883609] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:83, local_retry_times:83, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:38.883627] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.883633] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:38.883637] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.883650] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:38.883703] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:38.887893] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.887927] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478887827 2023-07-23 03:04:38.887827|[process_begin] u=0 in_queue_time:3663137, receive_ts:1690081475224690, enqueue_ts:1690081478887820|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=24 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=1 |[parse_job_end] u=4 |[remote_task_completed] u=36 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29241}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=9 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=2 |[close_plan_begin] u=1 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=2 |[query_end] u=8 |[process_end] u=0 run_ts:1690081478887829|total_timeu=99)
[2023-07-23 11:04:38.891892] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.891913] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.891955] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=13376552, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.891964] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.891975] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.891974] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.892002] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13485237, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.892000] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1001}, delta=13484999, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.892012] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.892014] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.892019] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.892030] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.892029] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=26] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.892034] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.892037] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get wrs ts(ls_id={id:1002}, delta=13814286, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.892049] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.892083] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=28] get wrs ts(ls_id={id:1003}, delta=13748427, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.892099] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.892163] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=5][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.892172] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.892180] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=4][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.892198] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=17][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.892213] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=15] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.892228] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=13] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:38.892234] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04D-0-0] [lt=4] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081478892232, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:38.892479] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BED-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.912671] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [3743][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=32] start do ls ha handler(ls_id_array_=[{id:1}])
[2023-07-23 11:04:38.923066] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:38.923078] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081478867496], now0=[mts=1690081478867496], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:38.923088] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:123, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.923112] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=23][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:38.923128] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=15][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:123, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.923147] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=18] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081478837154, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:123, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:38.923179] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=31][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:38.923189] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:38.923194] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:38.923200] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:38.923204] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:38.923210] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=120, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:38.923218] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8] will sleep(sleep_us=100000, remain_us=16246629, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=120, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:38.927195] INFO  [PALF] check_parent_health (log_config_mgr.cpp:2351) [3632][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=36] re_register_parent reason(palf_id=1, self="192.168.1.1:1001", is_registering_timeout=false, first_registration=true, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=-1, last_first_register_time_us=1690081478926477, register_time_us=-1, curr_time_us=1690081478927192)
[2023-07-23 11:04:38.927210] WDIAG [PALF] check_parent_health (log_config_mgr.cpp:2354) [3632][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4023] register request timeout, re_register_parent failed(ret=-4023, ret="OB_EAGAIN", palf_id=1, self="192.168.1.1:1001")
[2023-07-23 11:04:38.928278] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=37][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:38.928288] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196743}, scn:1690081478891751}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196743}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478817186, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.928318] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=30][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196743}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478817186, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.928436] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=117][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:38.928441] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.928445] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:38.928449] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:38.928452] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:38.928455] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:38.928458] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:38.928464] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503971, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196743}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478817186, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196743}, scn:1690081478891751}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:38.928763] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=298][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:38.928767] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:38.928773] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:38.928776] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=2][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:38.928780] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:38.928783] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=2][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:38.928787] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:38.928790] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:38.928793] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:38.928796] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:38.928799] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=2][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:38.928802] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=2][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:38.928819] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:54, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:38.928831] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196743}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478817186, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.928854] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=23] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196743}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478817186, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.928873] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=18] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081478891751, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196743}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478817186, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:38.928896] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=19][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:38.928903] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:38.928906] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:38.928910] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=94346329)
[2023-07-23 11:04:38.928923] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5] [packet retry query](TRACE=begin_ts=1690081478928194 2023-07-23 03:04:38.928194|[process_begin] u=0 in_queue_time:5652955, receive_ts:1690081473275239, enqueue_ts:1690081478928181|[start_sql] u=1 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=38 |[check_priv] u=7 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=1 |[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[insert_open] u=1 |[insert_start] u=9 |[get_das_id_begin] u=10 |[get_das_id_end] u=0 |[insert_end] u=2 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=190 |[sync_auto_value] u=320 |[close_plan_begin] u=43 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=64 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=27 |[process_end] u=0 run_ts:1690081478928197|total_timeu=728)
[2023-07-23 11:04:38.929825] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.929855] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=4][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:38.930844] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EE-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.931839] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.932823] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.937484] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081478937473]})
[2023-07-23 11:04:38.937499] INFO  [STORAGE.TRANS] operator() (ob_ts_mgr.h:171) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=-4122, ret="OB_RPC_POST_ERROR", gts_tenant_info={v:1})
[2023-07-23 11:04:38.937505] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.937510] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.937511] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081478937505]})
[2023-07-23 11:04:38.937520] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.937524] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.937671] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=4][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.937680] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.937684] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081478937676]})
[2023-07-23 11:04:38.937688] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=4][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.937705] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=16][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.937718] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=12] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.937733] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=12] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:38.937740] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04E-0-0] [lt=5] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081478937738, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:38.937848] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E7-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.938163] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C2C-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.938179] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C2C-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.938317] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=25] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081478938312})
[2023-07-23 11:04:38.938324] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC23-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:38.938776] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.938862] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478938261 2023-07-23 03:04:38.938261|[process_begin] u=0 in_queue_time:2984941, receive_ts:1690081475953320, enqueue_ts:1690081478938251|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=39 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=1 |[plc_serialize_begin] u=0 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=18 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=1 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=9 |[do_open_plan_end] u=1 |[get_das_id_begin] u=41 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=14 |[calc_partition_location_begin] u=344 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:427|[do_open_plan_begin] u=5 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=28 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[close_plan_begin] u=18 |[close_das_task_begin] u=0 |[close_das_task_end] u=19 |[start_end_stmt] u=18 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=11 |[process_end] u=1 run_ts:1690081478938263|total_timeu=600)
[2023-07-23 11:04:38.939784] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:38.939797] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081478883695], now0=[mts=1690081478883695], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:38.939809] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908671}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478883318, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.939839] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=28][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:38.939854] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=15][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908671}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478883318, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.939878] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=23] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081478837157, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908671}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478883318, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:38.939917] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=39][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:38.939928] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:38.939933] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:38.939938] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:38.939944] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:38.939955] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] will sleep(sleep_us=84000, remain_us=21736583, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=84, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:38.943995] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] Cache replace map node details(ret=0, replace_node_count=8, replace_time=10590, replace_start_pos=21900000, replace_num=100000)
[2023-07-23 11:04:38.951251] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1944-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.956516] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.956543] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13494680, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.956554] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.956795] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.956809] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.956817] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.956830] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.956836] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.956970] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.956980] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.956992] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=7][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.957014] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=21][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.957034] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=19] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.957054] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=16] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:38.957537] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.968938] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.969023] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081478968523 2023-07-23 03:04:38.968523|[process_begin] u=0 in_queue_time:2944697, receive_ts:1690081476023826, enqueue_ts:1690081478968515|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=23 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=1 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_das_id_begin] u=2 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=13 |[calc_partition_location_begin] u=336 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=0 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=5 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=24 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=40 |[close_das_task_begin] u=0 |[close_das_task_end] u=18 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=7 |[process_end] u=0 run_ts:1690081478968527|total_timeu=499)
[2023-07-23 11:04:38.971166] INFO  [PALF] is_pending_log_clear_ (log_state_mgr.cpp:747) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=16] follower pending log not clear, need wait(bool_ret=false, palf_id=1003, last_slide_end_lsn={lsn:375365428166}, pending_end_lsn={lsn:375365428400})
[2023-07-23 11:04:38.974133] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3633][T1_LogUpdater][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.974170] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.974178] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.974188] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:38.974192] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=3][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.974328] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:38.974336] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:38.974344] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=4][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.974361] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=16][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.974375] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=13] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:38.974390] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=13] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:38.974398] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B04F-0-0] [lt=6] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081478974396, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:38.978725] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=18][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:38.978733] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196744}, scn:1690081478973829}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196744}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478867534, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.978758] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196744}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478867534, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:38.978872] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=114][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:38.978877] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.978904] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196744}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478867534, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.978926] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=21] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196744}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478867534, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:38.978946] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=19] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081478973829, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196744}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478867534, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:38.978977] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478978642 2023-07-23 03:04:38.978642|[process_begin] u=0 in_queue_time:6318506, receive_ts:1690081472660135, enqueue_ts:1690081478978632|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=39 |[check_priv] u=7 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=2 |[value_after_calc_row] u=7 |[insert_start] u=1 |[get_das_id_begin] u=9 |[get_das_id_end] u=1 |[insert_end] u=2 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=160 |[sync_auto_value] u=2 |[close_plan_begin] u=16 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=64 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=8 |[process_end] u=1 run_ts:1690081478978644|total_timeu=335)
[2023-07-23 11:04:38.979775] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793ED-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.980703] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793ED-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.981593] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793ED-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.982490] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793ED-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.988734] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=34][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478974396})
[2023-07-23 11:04:38.988747] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478974396}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:38.988760] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:38.988765] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:38.988770] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478974396}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:38.988788] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:38.988798] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:38.988801] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:38.988805] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=2][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:38.989069] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.989100] INFO  [CLOG] is_in_sync_ (palf_handle_impl.cpp:4199) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=0] is_in_sync(ret=-4122, palf_id=1, is_in_sync=false, is_use_cache=false, remote_check_is_log_sync=false, leader_max_scn={val:18446744073709551615}, local_max_scn={val:1690081465515197326})
[2023-07-23 11:04:38.989277] INFO  [CLOG] handle_sync_request (log_request_handler.cpp:486) [83649][T1002_L0_G1][T1002][YB420A01819C-0005FDE7216A81B3-0-0] [lt=29] i am not leader(ret=-4038, palf_id=1003, req={get_type:1}, role=2, is_pending_state=true)
[2023-07-23 11:04:38.989397] INFO  [CLOG] is_in_sync_ (palf_handle_impl.cpp:4199) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=0] is_in_sync(ret=-4038, palf_id=1003, is_in_sync=false, is_use_cache=false, remote_check_is_log_sync=false, leader_max_scn={val:18446744073709551615}, local_max_scn={val:1690081465143322589})
[2023-07-23 11:04:38.989462] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C1D-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.989478] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C1D-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.989497] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3A-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:38.989510] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3A-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.990001] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:38.990037] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=16][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478974396})
[2023-07-23 11:04:38.990052] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=16][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081478974396}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:38.990065] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:38.990070] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:38.990078] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:38.990082] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:38.990085] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:38.990089] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=2][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:38.990096] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:38.990099] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:38.990102] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=2][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:38.990105] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=2][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:38.990108] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:38.990111] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:38.990116] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=2][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:38.990122] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:38.990126] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:38.990130] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:38.990135] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:38, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:38.990143] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:38.990148] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=6234542)
[2023-07-23 11:04:38.990136] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BEE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:38.990168] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541947861873, cache_obj->added_lc()=false, cache_obj->get_object_id()=20495, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:38.990184] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081478988706 2023-07-23 03:04:38.988706|[process_begin] u=0 in_queue_time:3764016, receive_ts:1690081475224690, enqueue_ts:1690081478988699|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=3 |[session] u=0 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=21 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[pc_choose_plan] u=72 |[transform_with_outline_begin] u=43 |[transform_with_outline_end] u=1 |[resolve_begin] u=2 |[resolve_end] u=400 |[transform_begin] u=17 |[optimize_begin] u=609 |[tl_calc_part_id_end] u=117 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=142 |[query_end] u=47 |[process_end] u=1 run_ts:1690081478988710|total_timeu=1477)
[2023-07-23 11:04:38.991954] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1}, delta=13476569, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.991973] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.991994] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1001}, delta=13585016, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.992006] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.992026] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get wrs ts(ls_id={id:1002}, delta=13914303, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.992038] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.992031] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:38.992058] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get wrs ts(ls_id={id:1003}, delta=13848444, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.992070] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.992080] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13585254, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:38.992097] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:38.998888] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=1][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.015324] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.023337] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.024092] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.029907] INFO  [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:132) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=17] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1002, epoch:-1, self:"192.168.1.1:1001", switch_to_leader_ts:-1})
[2023-07-23 11:04:39.029920] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.037509] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C7F-0-0] [lt=16] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1690081479037496})
[2023-07-23 11:04:39.037521] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC24-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.037540] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3291][][T0][YB420A01819C-0005FDE715FDDC24-0-0] [lt=16][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.038056] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.038076] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479038052]})
[2023-07-23 11:04:39.038094] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479038088]})
[2023-07-23 11:04:39.038120] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.038136] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=14][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.038158] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.038169] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.038285] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479038276]})
[2023-07-23 11:04:39.038346] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=14][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.038368] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=21][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.038389] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=13][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.038439] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E8-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.038427] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=36][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.038455] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=27] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.038488] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=29] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.038504] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B050-0-0] [lt=11] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479038501, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.039010] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C2D-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.039212] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=18] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479039205})
[2023-07-23 11:04:39.039218] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC25-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.039333] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.039394] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196745}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478928585, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.039439] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=44] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196745}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478928585, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.039474] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=34] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479038194, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196745}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478928585, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.039532] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479039186 2023-07-23 03:04:39.039186|[process_begin] u=0 in_queue_time:5763946, receive_ts:1690081473275239, enqueue_ts:1690081479039173|[start_sql] u=1 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=7 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=63 |[check_priv] u=12 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=3 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=1 |[insert_open] u=2 |[value_start_calc_row] u=3 |[value_after_calc_row] u=19 |[insert_start] u=0 |[get_das_id_begin] u=16 |[get_das_id_end] u=0 |[insert_end] u=4 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=28 |[sync_auto_value] u=4 |[close_plan_begin] u=28 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=122 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=14 |[process_end] u=1 run_ts:1690081479039194|total_timeu=345)
[2023-07-23 11:04:39.039786] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.039830] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931C-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.039845] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931C-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.039895] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479039151 2023-07-23 03:04:39.039151|[process_begin] u=0 in_queue_time:3085831, receive_ts:1690081475953320, enqueue_ts:1690081479039143|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=41 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=34 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=2 |[check_priv] u=2 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=1 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=5 |[do_open_plan_end] u=0 |[get_das_id_begin] u=60 |[get_das_id_end] u=1 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=14 |[calc_partition_location_begin] u=429 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:427|[do_open_plan_begin] u=7 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[remote_task_completed] u=29 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=22 |[close_das_task_begin] u=1 |[close_das_task_end] u=27 |[start_end_stmt] u=20 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=14 |[process_end] u=1 run_ts:1690081479039153|total_timeu=743)
[2023-07-23 11:04:39.040304] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931C-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.040745] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931C-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.041189] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931C-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.041416] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC26-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.041409] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=25] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1690081479041398})
[2023-07-23 11:04:39.042436] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35C-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1690081479042430})
[2023-07-23 11:04:39.042441] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC27-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.043554] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=16] new mysql sessid created(conn.sessid_=3222112456, support_ssl=false)
[2023-07-23 11:04:39.043611] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=22] sm conn init succ(conn.sessid_=3222112456, sess.client_addr_="127.0.0.1:57962")
[2023-07-23 11:04:39.043632] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=15] accept one succ(*s={this:0x7fd5405fd290, fd:360, err:0, last_decode_time_:0, last_write_time_:1690081479043609, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.043713] INFO  [SERVER] extract_user_tenant (obmp_connect.cpp:1243) [3192][MysqlQueueTh0][T0][Y0-0005FDE6FD2C2EC7-0-0] [lt=41] username and tenantname(user_name=ocp_monitor, tenant_name=)
[2023-07-23 11:04:39.043800] INFO  load_privilege_info (obmp_connect.cpp:426) [3192][MysqlQueueTh0][T1][Y0-0005FDE6FD2C2EC7-0-0] [lt=22] no tenant name set, use default tenant name(tenant_name=sys)
[2023-07-23 11:04:39.044467] INFO  [SERVER] process (obmp_connect.cpp:369) [3192][MysqlQueueTh0][T1][Y0-0005FDE6FD2C2EC7-0-0] [lt=15] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222112456, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:39.044650] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3769][T1_L0_G0][T1][YB420A01819C-0005FDE72ED825D3-0-0] [lt=6] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1690081479044644})
[2023-07-23 11:04:39.044656] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC28-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.045740] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3222102659, support_ssl=false)
[2023-07-23 11:04:39.045789] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=15] sm conn init succ(conn.sessid_=3222102659, sess.client_addr_="127.0.0.1:57964")
[2023-07-23 11:04:39.045805] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=12] accept one succ(*s={this:0x7fd4859641c0, fd:386, err:0, last_decode_time_:0, last_write_time_:1690081479045787, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.045881] INFO  [SERVER] extract_user_tenant (obmp_connect.cpp:1243) [3193][MysqlQueueTh1][T0][Y0-0005FDE6FA9C6EDA-0-0] [lt=15] username and tenantname(user_name=ocp_monitor, tenant_name=)
[2023-07-23 11:04:39.045942] INFO  load_privilege_info (obmp_connect.cpp:426) [3193][MysqlQueueTh1][T1][Y0-0005FDE6FA9C6EDA-0-0] [lt=10] no tenant name set, use default tenant name(tenant_name=sys)
[2023-07-23 11:04:39.046302] INFO  [SERVER] process (obmp_connect.cpp:369) [3193][MysqlQueueTh1][T1][Y0-0005FDE6FA9C6EDA-0-0] [lt=8] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222102659, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:39.046503] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3770][T1_L0_G0][T1][YB420A01819C-0005FDE72CD90294-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1690081479046496})
[2023-07-23 11:04:39.046515] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC29-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.051349] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=21][errcode=-4009] open file error(filename="cgroup/tenant_1/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:39.051383] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=35][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1/cpuacct.usage", usage_value="", tenant_id=1)
[2023-07-23 11:04:39.051681] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35C-0-0] [lt=11][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:39.051875] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1945-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.051894] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1945-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.052336] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [3769][T1_L0_G0][T1][YB420A01819C-0005FDE72ED825D3-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:39.052528] INFO  [SERVER] process (obmp_quit.h:55) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35D-0-0] [lt=16] quit
[2023-07-23 11:04:39.052520] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd5405fd290, fd:360, err:5, last_decode_time_:1690081479052506, last_write_time_:1690081479052444, read_buffer_.get_consume_sz():346, get_pending_flag():1, get_trace_id():YB420A01819C-0005FDE72D09E35D-0-0})
[2023-07-23 11:04:39.052536] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:152) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35D-0-0] [lt=8][errcode=-4009] sock has error(ret=-4009)
[2023-07-23 11:04:39.052541] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:929) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35D-0-0] [lt=5][errcode=-4009] write response fail(ret=-4009)
[2023-07-23 11:04:39.052547] WDIAG [SERVER] response (obmp_base.cpp:86) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35D-0-0] [lt=6][errcode=-4009] failed to flush_buffer(ret=-4009)
[2023-07-23 11:04:39.052551] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:48) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35D-0-0] [lt=3][errcode=-4009] response rpc result fail(ret=-4009)
[2023-07-23 11:04:39.052547] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=20] kill and revert session(conn.sessid_=3222112456, proxy_sessid=0, server_id=3, ret=0)
[2023-07-23 11:04:39.052561] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=12] can close safely, do destroy(*s={this:0x7fd5405fd290, fd:360, err:5, last_decode_time_:1690081479052506, last_write_time_:1690081479052444, read_buffer_.get_consume_sz():351, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.052586] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=16] unlock session of tenant(conn.sessid_=3222112456, proxy_sessid=0, conn.tenant_id_=1)
[2023-07-23 11:04:39.052599] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=13] connection close(sessid=3222112456, proxy_sessid=0, tenant_id=1, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:39.052627] INFO  [SERVER] run (obmp_disconnect.cpp:78) [3768][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] free session successfully(sessid=3222112456)
[2023-07-23 11:04:39.053371] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1001][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=13][errcode=-4009] open file error(filename="cgroup/tenant_1001/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:39.053407] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1001][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=37][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1001/cpuacct.usage", usage_value="", tenant_id=1001)
[2023-07-23 11:04:39.054187] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=20] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080742373882309}, clog_base_lsn:{lsn:8388096000}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831059622276}, all_id_meta:{id_meta:[{limited_id:1690081474308899777, latest_log_ts:{val:1690081457643664628}}, {limited_id:834000001, latest_log_ts:{val:1690081371645352093}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:39.054237] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=48] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1001}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741701524066}, clog_base_lsn:{lsn:703996121088}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831251246577}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:39.054267] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=29] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1002}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741745971437}, clog_base_lsn:{lsn:406789103616}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831214220192}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:39.054302] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=33] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1003}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741730241082}, clog_base_lsn:{lsn:375249862656}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831245549869}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:1, proposal_id:54, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:39.054604] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.054623] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=19][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081572900748, now=[mts=1690081478998879], now0=[mts=1690081478998879], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.054637] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=14][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478883318, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.054678] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=35][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:1003}, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60})
[2023-07-23 11:04:39.054696] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=18][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=3222118872, *tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478883318, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.054722] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=24] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081478883318, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.054774] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=51][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:501113, schema_version:1689916332789704, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.054786] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.054792] WDIAG [SQL] open (ob_result_set.cpp:157) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=5][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.054800] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=5][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:37, local_retry_times:0, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:2, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.054813] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] result set open failed, check if need retry(ret=-4283, cli_ret=-4283, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:39.054846] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081478998831 2023-07-23 03:04:38.998831|[process_begin] u=0 in_queue_time:6028082, receive_ts:1690081472970748, enqueue_ts:1690081478998821|[start_sql] u=0 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=30 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=3 |[check_priv] u=3 |[plan_id] u=1 plan_id:45401|[do_open_plan_begin] u=1 plan_id:45401|[sql_start_stmt_begin] u=1 |[post_packet] u=8 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=55900 |[do_open_plan_end] u=6 |[close_plan_begin] u=30 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=5 |[query_end] u=16 |[process_end] u=1 run_ts:1690081478998834|total_timeu=56013)
[2023-07-23 11:04:39.056145] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1002][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=12][errcode=-4009] open file error(filename="cgroup/tenant_1002/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:39.056164] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1002][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=20][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1002/cpuacct.usage", usage_value="", tenant_id=1002)
[2023-07-23 11:04:39.056567] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.056617] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13594912, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.056636] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.057124] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:264) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C80-0-0] [lt=9][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:39.057350] INFO  [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1331) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1)
[2023-07-23 11:04:39.057365] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1022) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=14] start lib cache evict(tenant_id=1, mem_hold=211812352, mem_limit=858993455, cache_obj_num=732, cache_node_num=729)
[2023-07-23 11:04:39.057375] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1039) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=9] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=211812352, mem_limit=858993455, cache_obj_num=732, cache_node_num=729)
[2023-07-23 11:04:39.057384] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2024) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:39.058163] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd2684b4030, fd:390, err:5, last_decode_time_:1690081479058148, last_write_time_:1690081479057260, read_buffer_.get_consume_sz():1751, get_pending_flag():1, get_trace_id():YB420A01819C-0005FDE72D1C9C81-0-0})
[2023-07-23 11:04:39.058179] INFO  [SERVER] process (obmp_quit.h:55) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C81-0-0] [lt=13] quit
[2023-07-23 11:04:39.058192] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:152) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C81-0-0] [lt=12][errcode=-4009] sock has error(ret=-4009)
[2023-07-23 11:04:39.058199] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=28] kill and revert session(conn.sessid_=3222113392, proxy_sessid=0, server_id=3, ret=0)
[2023-07-23 11:04:39.058203] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:929) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C81-0-0] [lt=12][errcode=-4009] write response fail(ret=-4009)
[2023-07-23 11:04:39.058212] WDIAG [SERVER] response (obmp_base.cpp:86) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C81-0-0] [lt=8][errcode=-4009] failed to flush_buffer(ret=-4009)
[2023-07-23 11:04:39.058222] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:48) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C81-0-0] [lt=10][errcode=-4009] response rpc result fail(ret=-4009)
[2023-07-23 11:04:39.060608] INFO  [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1799) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] Dumping All Cache Objs(alloc_obj_list.count()=732, alloc_obj_list=[{obj_id:1, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109325}, {obj_id:2, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111572}, {obj_id:3, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:341289}, {obj_id:4, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235127}, {obj_id:5, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184447}, {obj_id:6, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229842}, {obj_id:7, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179161}, {obj_id:8, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123908}, {obj_id:9, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128433}, {obj_id:11, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116945}, {obj_id:13, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129603}, {obj_id:15, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117454}, {obj_id:16, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109248}, {obj_id:17, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129226}, {obj_id:18, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109760}, {obj_id:19, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122149}, {obj_id:20, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114037}, {obj_id:21, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158335}, {obj_id:22, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121145}, {obj_id:23, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128430}, {obj_id:26, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226632}, {obj_id:29, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115569}, {obj_id:37, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145358}, {obj_id:38, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165873}, {obj_id:40, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127310}, {obj_id:41, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146874}, {obj_id:42, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131649}, {obj_id:43, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131770}, {obj_id:44, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125898}, {obj_id:45, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145121}, {obj_id:46, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130841}, {obj_id:47, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131994}, {obj_id:48, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121918}, {obj_id:49, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119714}, {obj_id:50, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118468}, {obj_id:51, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132696}, {obj_id:52, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125866}, {obj_id:53, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121229}, {obj_id:54, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118644}, {obj_id:55, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121560}, {obj_id:56, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118676}, {obj_id:57, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121565}, {obj_id:58, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118621}, {obj_id:59, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124750}, {obj_id:60, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120121}, {obj_id:61, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116266}, {obj_id:62, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141345}, {obj_id:63, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116282}, {obj_id:64, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120980}, {obj_id:65, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116682}, {obj_id:66, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124583}, {obj_id:67, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116330}, {obj_id:68, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117450}, {obj_id:69, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239802}, {obj_id:70, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187352}, {obj_id:73, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223151}, {obj_id:74, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170701}, {obj_id:76, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113692}, {obj_id:78, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125220}, {obj_id:81, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116890}, {obj_id:82, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115894}, {obj_id:84, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111136}, {obj_id:112, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121899}, {obj_id:113, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159037}, {obj_id:114, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118845}, {obj_id:116, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88994}, {obj_id:117, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151029}, {obj_id:118, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117628}, {obj_id:119, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118807}, {obj_id:120, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88832}, {obj_id:121, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:124, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109685}, {obj_id:135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170252}, {obj_id:139, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157877}, {obj_id:152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133335}, {obj_id:155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113973}, {obj_id:156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195192}, {obj_id:157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119284}, {obj_id:161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109728}, {obj_id:162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108544}, {obj_id:164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115837}, {obj_id:165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111259}, {obj_id:167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137904}, {obj_id:171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118032}, {obj_id:172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117558}, {obj_id:173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:306152}, {obj_id:174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108552}, {obj_id:175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112192}, {obj_id:176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109808}, {obj_id:177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111256}, {obj_id:178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112154}, {obj_id:179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112278}, {obj_id:180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147332}, {obj_id:181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120509}, {obj_id:184, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112479}, {obj_id:185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118190}, {obj_id:186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225682}, {obj_id:187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128216}, {obj_id:189, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120694}, {obj_id:190, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152472}, {obj_id:192, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124689}, {obj_id:194, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121903}, {obj_id:195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107757}, {obj_id:197, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110196}, {obj_id:198, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106208}, {obj_id:200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126038}, {obj_id:201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120794}, {obj_id:202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112445}, {obj_id:203, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138373}, {obj_id:204, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124081}, {obj_id:206, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110192}, {obj_id:213, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:373654}, {obj_id:214, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:505828}, {obj_id:215, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113398}, {obj_id:220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131588}, {obj_id:221, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144772}, {obj_id:230, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107418}, {obj_id:232, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115072}, {obj_id:233, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:234, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:93529}, {obj_id:235, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123553}, {obj_id:236, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116626}, {obj_id:238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121199}, {obj_id:239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106928}, {obj_id:240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112090}, {obj_id:264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:90440}, {obj_id:266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135852}, {obj_id:267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134273}, {obj_id:269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145191}, {obj_id:273, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123608}, {obj_id:274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108417}, {obj_id:278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111783}, {obj_id:305, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121676}, {obj_id:306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141832}, {obj_id:311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129452}, {obj_id:312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115712}, {obj_id:313, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124673}, {obj_id:322, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88886}, {obj_id:324, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140374}, {obj_id:328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113091}, {obj_id:330, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159280}, {obj_id:331, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125300}, {obj_id:332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117330}, {obj_id:337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123557}, {obj_id:345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111769}, {obj_id:346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123643}, {obj_id:356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125880}, {obj_id:361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112752}, {obj_id:362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110232}, {obj_id:363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108568}, {obj_id:364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113753}, {obj_id:365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119640}, {obj_id:402, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:857901}, {obj_id:403, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:452223}, {obj_id:411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110518}, {obj_id:421, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121605}, {obj_id:422, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128289}, {obj_id:426, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128217}, {obj_id:427, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107111}, {obj_id:429, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108426}, {obj_id:437, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117618}, {obj_id:438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138464}, {obj_id:439, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129287}, {obj_id:440, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:105488}, {obj_id:443, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115530}, {obj_id:447, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197761}, {obj_id:450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199814}, {obj_id:452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207533}, {obj_id:474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133309}, {obj_id:522, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:375064}, {obj_id:524, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480101}, {obj_id:534, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117040}, {obj_id:580, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122952}, {obj_id:605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:345007}, {obj_id:660, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223440}, {obj_id:661, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115072}, {obj_id:665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118024}, {obj_id:666, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116400}, {obj_id:667, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114060}, {obj_id:802, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120968}, {obj_id:855, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121960}, {obj_id:862, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114399}, {obj_id:863, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118344}, {obj_id:869, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117119}, {obj_id:876, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150075}, {obj_id:877, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175815}, {obj_id:925, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195664}, {obj_id:933, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129181}, {obj_id:944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132503}, {obj_id:945, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137742}, {obj_id:946, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126636}, {obj_id:948, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120424}, {obj_id:955, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127224}, {obj_id:956, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116704}, {obj_id:957, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114720}, {obj_id:1014, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173857}, {obj_id:1015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132988}, {obj_id:1016, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129138}, {obj_id:1083, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124264}, {obj_id:1145, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130168}, {obj_id:1146, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122296}, {obj_id:1169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280049}, {obj_id:1266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88881}, {obj_id:1269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:11552}, {obj_id:1271, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:916091}, {obj_id:1274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:1275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107490}, {obj_id:1276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110398}, {obj_id:1278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124488}, {obj_id:1279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121664}, {obj_id:1283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106461}, {obj_id:1284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142870}, {obj_id:1285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175250}, {obj_id:1286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161389}, {obj_id:1287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137813}, {obj_id:1288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208605}, {obj_id:1289, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156033}, {obj_id:1290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159046}, {obj_id:1291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191322}, {obj_id:1297, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144114}, {obj_id:1298, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131804}, {obj_id:1299, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167182}, {obj_id:1300, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140586}, {obj_id:1301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237139}, {obj_id:1302, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162351}, {obj_id:1303, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220896}, {obj_id:1306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143476}, {obj_id:1307, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164310}, {obj_id:1308, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121121}, {obj_id:1309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220799}, {obj_id:1310, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142485}, {obj_id:1311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204086}, {obj_id:1315, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153942}, {obj_id:1316, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132418}, {obj_id:1317, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213169}, {obj_id:1318, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176390}, {obj_id:1319, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:428742}, {obj_id:1320, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208446}, {obj_id:1321, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:420997}, {obj_id:1325, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147492}, {obj_id:1326, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182923}, {obj_id:1327, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:811052}, {obj_id:1328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:300175}, {obj_id:1329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:967217}, {obj_id:1330, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286896}, {obj_id:1334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143185}, {obj_id:1335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131501}, {obj_id:1336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162857}, {obj_id:1337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:411417}, {obj_id:1338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216472}, {obj_id:1339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:488389}, {obj_id:1340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199461}, {obj_id:1345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145055}, {obj_id:1346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169934}, {obj_id:1347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126655}, {obj_id:1348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248008}, {obj_id:1349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149096}, {obj_id:1350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238697}, {obj_id:1353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132741}, {obj_id:1354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171395}, {obj_id:1355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251238}, {obj_id:1356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165610}, {obj_id:1359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145342}, {obj_id:1360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172824}, {obj_id:1361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257085}, {obj_id:1362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241945}, {obj_id:1367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146300}, {obj_id:1368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177171}, {obj_id:1369, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274985}, {obj_id:1370, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260698}, {obj_id:1372, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:466085}, {obj_id:1373, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:554863}, {obj_id:1380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132111}, {obj_id:1381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:148619}, {obj_id:1382, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175303}, {obj_id:1385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143827}, {obj_id:1386, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165757}, {obj_id:1387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132159}, {obj_id:1388, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226691}, {obj_id:1389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155636}, {obj_id:1390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210276}, {obj_id:1400, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153845}, {obj_id:1522, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132816}, {obj_id:1538, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:296281}, {obj_id:1641, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146892}, {obj_id:1642, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:1643, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283433}, {obj_id:1644, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275590}, {obj_id:1645, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142583}, {obj_id:1648, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143372}, {obj_id:1669, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:468828}, {obj_id:1686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152827}, {obj_id:1781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:380248}, {obj_id:1782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:517009}, {obj_id:1841, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117704}, {obj_id:2349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122600}, {obj_id:2362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111073}, {obj_id:2363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112556}, {obj_id:2365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140294}, {obj_id:2366, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133478}, {obj_id:2394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260616}, {obj_id:2431, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117352}, {obj_id:2451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121272}, {obj_id:2452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:2491, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151697}, {obj_id:2500, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135218}, {obj_id:2502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145948}, {obj_id:2503, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175710}, {obj_id:2504, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269768}, {obj_id:2505, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:255176}, {obj_id:2515, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180033}, {obj_id:2516, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182644}, {obj_id:2517, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:291295}, {obj_id:2518, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225595}, {obj_id:2519, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141982}, {obj_id:2535, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163630}, {obj_id:2536, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193037}, {obj_id:2538, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:558139}, {obj_id:2561, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156119}, {obj_id:2687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123272}, {obj_id:2712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:335251}, {obj_id:2864, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:357639}, {obj_id:2868, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:453965}, {obj_id:2924, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139696}, {obj_id:3119, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124568}, {obj_id:3201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134402}, {obj_id:3202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137472}, {obj_id:3203, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132350}, {obj_id:3275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138117}, {obj_id:3276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127600}, {obj_id:3277, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107922}, {obj_id:3279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110460}, {obj_id:3280, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116675}, {obj_id:3283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:3284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118296}, {obj_id:3285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137904}, {obj_id:3286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112502}, {obj_id:3288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113940}, {obj_id:3290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106432}, {obj_id:3291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:3293, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118680}, {obj_id:3294, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107219}, {obj_id:3295, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127909}, {obj_id:3309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:390507}, {obj_id:3473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130856}, {obj_id:3480, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151736}, {obj_id:3498, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319008}, {obj_id:3748, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138080}, {obj_id:3749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115376}, {obj_id:3938, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125896}, {obj_id:3993, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135764}, {obj_id:4060, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249784}, {obj_id:4061, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109801}, {obj_id:4062, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:4063, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:4064, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4065, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:4066, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4067, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88838}, {obj_id:4068, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4069, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4072, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4073, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88839}, {obj_id:4074, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4075, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4079, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133294}, {obj_id:4080, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:4081, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107168}, {obj_id:4084, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88964}, {obj_id:4195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:333559}, {obj_id:4200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:416289}, {obj_id:4267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131792}, {obj_id:4268, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116016}, {obj_id:4301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286580}, {obj_id:4518, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126536}, {obj_id:4546, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:4547, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4548, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4549, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88839}, {obj_id:4550, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4551, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112431}, {obj_id:4556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4568, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:299521}, {obj_id:4677, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173951}, {obj_id:4678, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202163}, {obj_id:4683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185458}, {obj_id:4684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219118}, {obj_id:4687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:792994}, {obj_id:4691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:805593}, {obj_id:4692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:960692}, {obj_id:4726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150855}, {obj_id:4866, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88952}, {obj_id:5048, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:259342}, {obj_id:5051, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:302866}, {obj_id:5079, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125592}, {obj_id:5128, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136112}, {obj_id:5287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:340096}, {obj_id:5291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:426188}, {obj_id:5364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127560}, {obj_id:5380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120632}, {obj_id:5381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124904}, {obj_id:5559, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:359785}, {obj_id:5561, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:456815}, {obj_id:5615, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257334}, {obj_id:5651, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136416}, {obj_id:5652, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174265}, {obj_id:5653, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129528}, {obj_id:5884, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234650}, {obj_id:5909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177506}, {obj_id:6176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142016}, {obj_id:6177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151189}, {obj_id:6178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120328}, {obj_id:6208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126232}, {obj_id:6385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274624}, {obj_id:6387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:329196}, {obj_id:6470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171259}, {obj_id:6578, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190649}, {obj_id:6579, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215440}, {obj_id:6584, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193411}, {obj_id:6590, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205876}, {obj_id:6631, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156453}, {obj_id:6774, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174901}, {obj_id:7027, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171911}, {obj_id:7060, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263837}, {obj_id:7191, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:344496}, {obj_id:7195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:433143}, {obj_id:7208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121656}, {obj_id:7263, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132480}, {obj_id:7264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150203}, {obj_id:7450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:278957}, {obj_id:7452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:332196}, {obj_id:7459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250592}, {obj_id:7462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290128}, {obj_id:7486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126920}, {obj_id:7547, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170957}, {obj_id:7806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:748217}, {obj_id:7807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:889404}, {obj_id:7813, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:904154}, {obj_id:7814, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1074659}, {obj_id:7882, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162719}, {obj_id:7921, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283317}, {obj_id:8073, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272428}, {obj_id:8075, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:322991}, {obj_id:8143, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141376}, {obj_id:8144, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145305}, {obj_id:8367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:346670}, {obj_id:8368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:436340}, {obj_id:8449, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167665}, {obj_id:8450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119976}, {obj_id:8483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276813}, {obj_id:8675, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270240}, {obj_id:8676, tenant_id:1, log_del_time:9223372036854775807, real_de
[2023-07-23 11:04:39.061584] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2032) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=974] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:39.070023] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.070110] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479069441 2023-07-23 03:04:39.069441|[process_begin] u=0 in_queue_time:3045615, receive_ts:1690081476023826, enqueue_ts:1690081479069432|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=36 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=4 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=23 |[calc_partition_location_begin] u=459 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=8 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=34 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[close_plan_begin] u=24 |[close_das_task_begin] u=0 |[close_das_task_end] u=26 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=9 |[process_end] u=1 run_ts:1690081479069444|total_timeu=668)
[2023-07-23 11:04:39.078305] INFO  [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:202) [83130][T1002_LCLSender][T1002][Y0-0000000000000000-0-0] [lt=20] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=23875, total_destructed_detector=23875, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x7fd4fa5ea2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0})
[2023-07-23 11:04:39.079147] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081478988600, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:124, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.079212] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=1] will sleep(sleep_us=100000, remain_us=16090636, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=121, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:39.079419] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B82-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.079446] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B82-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.079860] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=16][errcode=-4283]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.079873] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=1] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081478988601, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908672}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479023425, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.079917] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] will sleep(sleep_us=85000, remain_us=21596621, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=85, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:39.080128] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [83145][T1002_LSArchive][T1002][YB420A01819C-0005FDE628078DB5-0-0] [lt=13] gc stale ls task succ
[2023-07-23 11:04:39.082708] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73134578892, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:39.083890] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=18] Wash time detail, (compute_wash_size_time=91, refresh_score_time=691, wash_time=473)
[2023-07-23 11:04:39.089873] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196746}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478978051, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.089911] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=38] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196746}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081478978051, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.089939] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479038196, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196746}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081478978051, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.089986] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479089648 2023-07-23 03:04:39.089648|[process_begin] u=0 in_queue_time:6429512, receive_ts:1690081472660135, enqueue_ts:1690081479089636|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=62 |[check_priv] u=12 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=3 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=6 |[exec_plan_begin] u=1 |[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[insert_open] u=2 |[insert_start] u=19 |[get_das_id_begin] u=22 |[get_das_id_end] u=0 |[insert_end] u=6 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=14 |[sync_auto_value] u=4 |[close_plan_begin] u=54 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=2 |[end_stmt] u=102 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=12 |[process_end] u=1 run_ts:1690081479089651|total_timeu=337)
[2023-07-23 11:04:39.090325] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EF-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.090341] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EF-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.090816] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EF-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.091275] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EF-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.091727] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793EF-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.092021] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.092071] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=13576733, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.092086] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.092092] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.092119] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13685418, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.092126] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=23] get wrs ts(ls_id={id:1001}, delta=13685180, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.092134] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.092143] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.092175] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1002}, delta=14014467, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.092192] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.092218] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1003}, delta=13948608, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.092235] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.099812] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.099862] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081479099711 2023-07-23 03:04:39.099711|[process_begin] u=0 in_queue_time:3875021, receive_ts:1690081475224690, enqueue_ts:1690081479099703|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=33 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=2 |[parse_job_end] u=7 |[remote_task_completed] u=59 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29243}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=14 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=2 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=11 |[process_end] u=1 run_ts:1690081479099714|total_timeu=150)
[2023-07-23 11:04:39.103463] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=19560, clean_start_pos=8000000, clean_num=200000)
[2023-07-23 11:04:39.105342] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [83382][T1001_LSArchive][T1001][YB420A01819C-0005FDE628A78DB5-0-0] [lt=13] gc stale ls task succ
[2023-07-23 11:04:39.111855] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.111869] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.111878] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.111892] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.111898] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.112119] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.112132] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=11][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.112148] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=7][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.112174] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=25][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.112197] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=21] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.112218] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=17] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.112231] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=6] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479112228, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.112736] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3284][AutoLSLoc][T1][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.113245] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3284][AutoLSLoc][T1001][YB420A01819C-0005FDE7200AAFA7-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.118129] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD5-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.118145] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.129243] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.129266] INFO  [PALF] do_fetch_log_ (log_sliding_window.cpp:1927) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=1] do_fetch_log_ finished(ret=0, palf_id=1002, self="192.168.1.1:1001", dest="10.1.129.155:1001", fetch_log_count=2048, fetch_start_lsn={lsn:406891815358}, prev_lsn={lsn:406891814580}, fetch_start_log_id=130559082, last_slide_log_id=130559081, fetch_log_size=41943040, accepted_mode_pid=1, last_fetch_req_time=1690081479129263, last_fetch_end_lsn={lsn:406933758398}, last_fetch_max_log_id=130561129, last_fetch_committed_end_lsn={lsn:18446744073709551615}, trigger_type=0, this={palf_id:1002, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:130559082, max_lsn:406891815781, max_scn:{val:1690081465085443984}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:406891815781}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:406891815358}, last_submit_end_lsn:{lsn:406891815781}, last_submit_log_id:130559082, last_submit_log_pid:1, max_flushed_lsn:{lsn:406891815358}, max_flushed_end_lsn:{lsn:406891815781}, max_flushed_log_pid:1, committed_end_lsn:{lsn:406891815358}, last_slide_log_id:130559081, last_slide_scn:{val:1690081465077463222}, last_slide_lsn:{lsn:406891814580}, last_slide_end_lsn:{lsn:406891815358}, last_slide_log_pid:1, last_slide_log_accum_checksum:2796067235, last_fetch_end_lsn:{lsn:406933758398}, last_fetch_max_log_id:130561129, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd45c242530})
[2023-07-23 11:04:39.129964] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.131091] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:351) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ======
[2023-07-23 11:04:39.131105] INFO  [PALF] get_disk_usage (palf_env_impl.cpp:781) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=6758, used(MB):=5348)
[2023-07-23 11:04:39.132089] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] succ to get rec scn(*this={this:0x7fd478e04030, ls_id:{id:1}, tenant_id:1001, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1690048801926620928}, last_push_gc_task_ts_:1690048803399922, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:39.132109] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=20] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd483fe6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1690048802126249}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd483fe6080, snapshot_version:{val:1690048802126249}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:39.132136] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=22] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:0}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:0}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1})
[2023-07-23 11:04:39.132151] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7fd4867ec1b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7fd4867f1290, table_type:1, memtable_head:244, memtable_tail:245, t3m:0x7fd4ba7e2030, tables:[null, null, null, null, 0x7fd483fd6080, null, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, tx_data_table:0x7fd4867f2650, ls_tablet_svr:0x7fd4867ec190, slice_allocator:0x7fd4867f2690}, min_rec_scn={val:1690067853855506884}, ls_id_={id:1})
[2023-07-23 11:04:39.132810] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=24] get rec log scn(service_type_=0, rec_log_ts={val:1690081456904556569})
[2023-07-23 11:04:39.132820] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=10] get rec log scn(service_type_=1, rec_log_ts={val:1690061231333212449})
[2023-07-23 11:04:39.132828] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=2, rec_log_ts={val:1690075632203527680})
[2023-07-23 11:04:39.132835] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690061231333212449}, checkpoint_scn_in_ls_meta={val:1690061231333212449}, ls_id={id:1}, service_type="TRANS_ID")
[2023-07-23 11:04:39.132846] INFO  [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:264) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=575073757, threshold=2126008811)
[2023-07-23 11:04:39.132857] INFO  [STORAGE] flush_if_need_ (ob_ls.cpp:1564) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] the ls no need flush to advance_checkpoint(get_ls_id()={id:1}, need_flush=false)
[2023-07-23 11:04:39.136303] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [3383][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=6] can close safely, do destroy(*s={this:0x7fd356516030, fd:374, err:5, last_decode_time_:1690081478135247, last_write_time_:1690081478116266, read_buffer_.get_consume_sz():659, get_pending_flag():2, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.136332] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3383][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=19] unlock session of tenant(conn.sessid_=3222104394, proxy_sessid=0, conn.tenant_id_=1)
[2023-07-23 11:04:39.136342] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3383][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=10] connection close(sessid=3222104394, proxy_sessid=0, tenant_id=1, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:39.136398] INFO  [SERVER] run (obmp_disconnect.cpp:78) [3767][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=7] free session successfully(sessid=3222104394)
[2023-07-23 11:04:39.138744] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479138724]})
[2023-07-23 11:04:39.138774] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.138774] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=1] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479138765]})
[2023-07-23 11:04:39.138785] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.138797] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.138803] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.138964] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=6][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.138975] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.138985] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=6][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.139007] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=21][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.139028] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=20] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.139048] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=18] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.139055] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B051-0-0] [lt=5] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479139053, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.139066] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479139053]})
[2023-07-23 11:04:39.139121] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1E9-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.139831] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C1E-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.140123] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=43] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479140117})
[2023-07-23 11:04:39.140133] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC2A-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.140238] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196747}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479039241, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.140273] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=36] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196747}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479039241, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.140308] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=33] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479138267, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196747}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479039241, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.140349] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479140092 2023-07-23 03:04:39.140092|[process_begin] u=0 in_queue_time:5864852, receive_ts:1690081473275239, enqueue_ts:1690081479140081|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=4 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=50 |[check_priv] u=9 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=3 |[value_after_calc_row] u=8 |[insert_start] u=0 |[get_das_id_begin] u=16 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=20 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=99 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=10 |[process_end] u=1 run_ts:1690081479140098|total_timeu=256)
[2023-07-23 11:04:39.140681] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.140691] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F0-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.140793] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479140059 2023-07-23 03:04:39.140059|[process_begin] u=0 in_queue_time:3186739, receive_ts:1690081475953320, enqueue_ts:1690081479140052|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=44 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=27 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=2 |[pc_choose_plan] u=2 |[check_priv] u=2 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=1 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=9 |[do_open_plan_end] u=0 |[get_das_id_begin] u=54 |[get_das_id_end] u=1 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=421 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=6 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=27 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=4 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=24 |[close_das_task_begin] u=2 |[close_das_task_end] u=25 |[start_end_stmt] u=23 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=15 |[process_end] u=1 run_ts:1690081479140061|total_timeu=733)
[2023-07-23 11:04:39.141162] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F0-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.141620] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F0-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.142064] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F0-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.146514] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [3382][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=5] can close safely, do destroy(*s={this:0x7fd45154c030, fd:361, err:5, last_decode_time_:1690081478146174, last_write_time_:1690081478113763, read_buffer_.get_consume_sz():351, get_pending_flag():2, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.146540] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3382][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=17] unlock session of tenant(conn.sessid_=3222113706, proxy_sessid=0, conn.tenant_id_=1)
[2023-07-23 11:04:39.146549] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3382][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=8] connection close(sessid=3222113706, proxy_sessid=0, tenant_id=1, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:39.146586] INFO  [SERVER] run (obmp_disconnect.cpp:78) [3767][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=20] free session successfully(sessid=3222113706)
[2023-07-23 11:04:39.152831] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1946-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.156609] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.156633] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=13694988, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.156644] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.160378] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.160541] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] Cache replace map node details(ret=0, replace_node_count=10, replace_time=16467, replace_start_pos=22000000, replace_num=100000)
[2023-07-23 11:04:39.165073] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.167069] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167087] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167098] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167108] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167118] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167127] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167137] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167149] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167159] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167168] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167176] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167185] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167195] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167204] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167214] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.167222] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.170788] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.170892] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479170332 2023-07-23 03:04:39.170332|[process_begin] u=0 in_queue_time:3146506, receive_ts:1690081476023826, enqueue_ts:1690081479170324|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=28 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=1 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=363 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=0 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=6 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=25 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=56 |[close_das_task_begin] u=0 |[close_das_task_end] u=20 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[query_end] u=7 |[process_end] u=0 run_ts:1690081479170335|total_timeu=559)
[2023-07-23 11:04:39.177382] INFO  [PALF] check_parent_health (log_config_mgr.cpp:2351) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=28] re_register_parent reason(palf_id=1, self="192.168.1.1:1001", is_registering_timeout=false, first_registration=true, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=-1, last_first_register_time_us=1690081479177258, register_time_us=-1, curr_time_us=1690081479177378)
[2023-07-23 11:04:39.177411] WDIAG [PALF] check_parent_health (log_config_mgr.cpp:2354) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=27][errcode=-4023] register request timeout, re_register_parent failed(ret=-4023, ret="OB_EAGAIN", palf_id=1, self="192.168.1.1:1001")
[2023-07-23 11:04:39.178010] INFO  [STORAGE] operator() (ob_tenant_freezer.cpp:125) [3641][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=14] ====== tenant freeze timer task ======
[2023-07-23 11:04:39.178457] INFO  [ARCHIVE] produce_log_fetch_task_ (ob_archive_sequencer.cpp:171) [3735][T1_ArcSeq][T1][YB420A01819C-0005FDE623678DB5-0-0] [lt=25] archive round not in doing status, just skip(key={incarnation:-1, dest_id:-1, round:-1}, state={status:"INVALID"})
[2023-07-23 11:04:39.179070] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3641][T1_Occam][T1][YB420A01819C-0005FDE71EE93C0E-0-0] [lt=22][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.179100] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3641][T1_Occam][T1][YB420A01819C-0005FDE71EE93C0E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.179206] WDIAG [STORAGE] check_and_freeze_normal_data_ (ob_tenant_freezer.cpp:408) [3641][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=0][errcode=0] [TenantFreezer] fail to do major freeze(tmp_ret=-4122)
[2023-07-23 11:04:39.179357] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.190534] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=36][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479139053})
[2023-07-23 11:04:39.190551] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=17][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479139053}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.190565] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.190571] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.190577] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479139053}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:39.190597] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=20][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:39.190610] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:39.190615] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:39.190621] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:39.192059] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:39.192076] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.192104] INFO  [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:808) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=false, can_update_version=false, cur_leader_epoch=0, start_service_tstamp_=0, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0)
[2023-07-23 11:04:39.192112] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=19][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479139053})
[2023-07-23 11:04:39.192128] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=16][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479139053}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.192136] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.192142] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.192150] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.192145] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=25] get wrs ts(ls_id={id:1}, delta=13676739, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.192154] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.192159] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:39.192162] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:39.192168] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.192168] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13785424, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.192161] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.192175] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:39.192179] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:39.192182] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:39.192178] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.192186] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:39.192191] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:39.192187] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1001}, delta=13785186, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.192194] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:39.192201] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:39.192207] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:39.192199] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.192211] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:39.192215] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:39.192219] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1002}, delta=14114473, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.192223] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:40, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:39.192230] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.192236] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=6032454)
[2023-07-23 11:04:39.192231] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.192251] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1003}, delta=14048614, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.192264] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541948063968, cache_obj->added_lc()=false, cache_obj->get_object_id()=20496, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:39.192265] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.192281] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081479190507 2023-07-23 03:04:39.190507|[process_begin] u=0 in_queue_time:3965816, receive_ts:1690081475224690, enqueue_ts:1690081479190499|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=21 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[pc_choose_plan] u=88 |[transform_with_outline_begin] u=79 |[transform_with_outline_end] u=1 |[resolve_begin] u=3 |[resolve_end] u=471 |[transform_begin] u=26 |[optimize_begin] u=697 |[tl_calc_part_id_end] u=151 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=174 |[query_end] u=57 |[process_end] u=1 run_ts:1690081479190509|total_timeu=1773)
[2023-07-23 11:04:39.196872] INFO  [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1331) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002)
[2023-07-23 11:04:39.196887] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1022) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1002, mem_hold=869539840, mem_limit=1063004405, cache_obj_num=1984, cache_node_num=1958)
[2023-07-23 11:04:39.196901] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1039) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=12] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=869539840, mem_limit=1063004405, cache_obj_num=1984, cache_node_num=1958)
[2023-07-23 11:04:39.196912] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2024) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=10] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:39.200754] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196748}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479089822, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.200789] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=35] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196748}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479089822, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.200815] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479138271, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196748}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479089822, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.200853] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479200624 2023-07-23 03:04:39.200624|[process_begin] u=0 in_queue_time:6540488, receive_ts:1690081472660135, enqueue_ts:1690081479200615|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=51 |[check_priv] u=8 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=1 |[insert_end] u=2 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=15 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=0 |[end_stmt] u=90 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=10 |[process_end] u=0 run_ts:1690081479200626|total_timeu=228)
[2023-07-23 11:04:39.201687] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EE-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.201935] INFO  do_work (ob_rl_mgr.cpp:704) [3225][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=20] swc wakeup.(stat_period_=1000000, ready=false)
[2023-07-23 11:04:39.200745] INFO  [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1799) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=9] Dumping All Cache Objs(alloc_obj_list.count()=1984, alloc_obj_list=[{obj_id:59, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109179}, {obj_id:436, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241752}, {obj_id:549, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113540}, {obj_id:560, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125156}, {obj_id:564, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110196}, {obj_id:565, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108544}, {obj_id:691, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112090}, {obj_id:692, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112210}, {obj_id:1367, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:1419, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:4212, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:4220, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:5192, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109728}, {obj_id:11057, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108552}, {obj_id:11059, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109808}, {obj_id:11060, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111256}, {obj_id:11194, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88952}, {obj_id:11319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:11320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:15478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140587}, {obj_id:15745, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126810}, {obj_id:15746, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140248}, {obj_id:15747, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128659}, {obj_id:27929, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142362}, {obj_id:27931, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128737}, {obj_id:27971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143089}, {obj_id:28026, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122421}, {obj_id:28581, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107590}, {obj_id:33091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128554}, {obj_id:34554, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:35697, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116836}, {obj_id:35720, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118097}, {obj_id:35721, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89647}, {obj_id:35722, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89453}, {obj_id:35811, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129607}, {obj_id:35814, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117342}, {obj_id:35888, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141527}, {obj_id:36341, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115569}, {obj_id:36991, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118097}, {obj_id:36992, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89647}, {obj_id:36993, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89453}, {obj_id:37327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134320}, {obj_id:37578, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132608}, {obj_id:39068, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143906}, {obj_id:39453, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107024}, {obj_id:39469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115590}, {obj_id:39684, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115949}, {obj_id:39841, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111363}, {obj_id:39878, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112304}, {obj_id:39926, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115580}, {obj_id:39930, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118242}, {obj_id:40799, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111572}, {obj_id:40807, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138778}, {obj_id:40808, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128342}, {obj_id:40809, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122922}, {obj_id:40810, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110504}, {obj_id:41224, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141493}, {obj_id:41415, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88881}, {obj_id:41418, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:11552}, {obj_id:41419, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:916091}, {obj_id:41420, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106461}, {obj_id:41421, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134402}, {obj_id:41422, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137472}, {obj_id:41423, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132350}, {obj_id:41425, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109997}, {obj_id:42005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119470}, {obj_id:42006, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140970}, {obj_id:42161, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243831}, {obj_id:42376, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121389}, {obj_id:42389, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231388}, {obj_id:42394, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243933}, {obj_id:42456, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237719}, {obj_id:42488, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242064}, {obj_id:42509, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227384}, {obj_id:42511, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193344}, {obj_id:42526, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136549}, {obj_id:42545, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191472}, {obj_id:42572, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143043}, {obj_id:42580, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237596}, {obj_id:42603, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147035}, {obj_id:42608, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239548}, {obj_id:42614, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235468}, {obj_id:42615, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226982}, {obj_id:42616, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239696}, {obj_id:42650, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235502}, {obj_id:42652, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235294}, {obj_id:42655, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195588}, {obj_id:42664, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250450}, {obj_id:42674, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233474}, {obj_id:42690, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136327}, {obj_id:42691, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141008}, {obj_id:42692, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237407}, {obj_id:42736, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235468}, {obj_id:42740, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189382}, {obj_id:42744, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176369}, {obj_id:42746, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122937}, {obj_id:42752, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235638}, {obj_id:42753, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229499}, {obj_id:42757, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233788}, {obj_id:42759, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231541}, {obj_id:42760, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235752}, {obj_id:42762, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239707}, {obj_id:42765, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233641}, {obj_id:42769, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233641}, {obj_id:42770, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231370}, {obj_id:42772, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225129}, {obj_id:42773, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227248}, {obj_id:42780, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229515}, {obj_id:42783, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229331}, {obj_id:42785, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237678}, {obj_id:42791, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225610}, {obj_id:42823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237576}, {obj_id:42847, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139794}, {obj_id:42848, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139842}, {obj_id:42869, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141707}, {obj_id:42877, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233515}, {obj_id:42929, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196382}, {obj_id:42930, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166832}, {obj_id:42933, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200801}, {obj_id:42934, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168438}, {obj_id:42975, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182017}, {obj_id:42976, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158792}, {obj_id:43125, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192454}, {obj_id:43126, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163561}, {obj_id:43135, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200583}, {obj_id:43136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168860}, {obj_id:43145, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198468}, {obj_id:43146, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167145}, {obj_id:43165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198417}, {obj_id:43166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167142}, {obj_id:43185, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225027}, {obj_id:43187, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194300}, {obj_id:43188, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165170}, {obj_id:43209, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123779}, {obj_id:43210, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115125}, {obj_id:43217, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231456}, {obj_id:43235, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250319}, {obj_id:43251, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192358}, {obj_id:43252, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164022}, {obj_id:43289, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192317}, {obj_id:43290, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163512}, {obj_id:43307, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231386}, {obj_id:43341, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190036}, {obj_id:43342, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162726}, {obj_id:43343, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188060}, {obj_id:43344, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161506}, {obj_id:43350, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175697}, {obj_id:43351, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:154660}, {obj_id:43371, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223010}, {obj_id:43374, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231405}, {obj_id:43412, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190369}, {obj_id:43413, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161908}, {obj_id:43440, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178268}, {obj_id:43468, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233465}, {obj_id:43469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185944}, {obj_id:43470, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160252}, {obj_id:43471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183905}, {obj_id:43472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159514}, {obj_id:43475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192081}, {obj_id:43476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163941}, {obj_id:43477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194553}, {obj_id:43478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163871}, {obj_id:43495, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186249}, {obj_id:43496, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160359}, {obj_id:43497, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190285}, {obj_id:43498, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162767}, {obj_id:43499, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194439}, {obj_id:43500, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164325}, {obj_id:43516, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235574}, {obj_id:43523, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235478}, {obj_id:43537, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198599}, {obj_id:43538, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166292}, {obj_id:43557, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198496}, {obj_id:43558, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167189}, {obj_id:43564, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179627}, {obj_id:43565, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157513}, {obj_id:43568, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200715}, {obj_id:43569, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168000}, {obj_id:43574, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190136}, {obj_id:43575, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162730}, {obj_id:43576, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182109}, {obj_id:43577, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157887}, {obj_id:43582, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231365}, {obj_id:43583, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198204}, {obj_id:43584, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167548}, {obj_id:43589, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241715}, {obj_id:43593, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200800}, {obj_id:43594, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167565}, {obj_id:43605, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194343}, {obj_id:43606, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164741}, {obj_id:43607, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180048}, {obj_id:43608, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157152}, {obj_id:43613, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227101}, {obj_id:43621, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195746}, {obj_id:43625, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196652}, {obj_id:43626, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165566}, {obj_id:43629, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222865}, {obj_id:43631, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237599}, {obj_id:43636, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250302}, {obj_id:43639, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190288}, {obj_id:43640, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162779}, {obj_id:43641, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145601}, {obj_id:43642, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138368}, {obj_id:43657, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:246169}, {obj_id:43660, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:43661, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160674}, {obj_id:43666, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222992}, {obj_id:43667, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225091}, {obj_id:43680, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237833}, {obj_id:43683, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196413}, {obj_id:43684, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166866}, {obj_id:43690, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192452}, {obj_id:43691, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163159}, {obj_id:43692, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200627}, {obj_id:43693, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167944}, {obj_id:43697, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188317}, {obj_id:43698, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161594}, {obj_id:43700, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243805}, {obj_id:43707, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184148}, {obj_id:43708, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159121}, {obj_id:43711, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:43712, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168429}, {obj_id:43713, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241821}, {obj_id:43714, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190251}, {obj_id:43715, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161845}, {obj_id:43716, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202618}, {obj_id:43717, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169167}, {obj_id:43720, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194445}, {obj_id:43721, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164779}, {obj_id:43741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188208}, {obj_id:43742, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161547}, {obj_id:43768, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190252}, {obj_id:43769, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162775}, {obj_id:43772, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200668}, {obj_id:43773, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168433}, {obj_id:43777, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234715}, {obj_id:43778, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183042}, {obj_id:43782, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231457}, {obj_id:43785, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196511}, {obj_id:43786, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165969}, {obj_id:43797, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235304}, {obj_id:43804, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190125}, {obj_id:43805, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162719}, {obj_id:43806, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198492}, {obj_id:43807, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167609}, {obj_id:43810, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196667}, {obj_id:43811, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166013}, {obj_id:43812, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192194}, {obj_id:43813, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163970}, {obj_id:43818, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194461}, {obj_id:43819, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165219}, {obj_id:43827, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186092}, {obj_id:43828, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160302}, {obj_id:43834, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186129}, {obj_id:43835, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160353}, {obj_id:43836, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180010}, {obj_id:43837, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157147}, {obj_id:43845, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179973}, {obj_id:43846, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156702}, {obj_id:43859, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181802}, {obj_id:43860, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157826}, {obj_id:43865, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227091}, {obj_id:43869, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158361}, {obj_id:43870, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150274}, {obj_id:43871, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231334}, {obj_id:43875, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227227}, {obj_id:43880, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198489}, {obj_id:43881, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167606}, {obj_id:43886, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198419}, {obj_id:43887, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167144}, {obj_id:43896, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188263}, {obj_id:43897, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160675}, {obj_id:43901, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196597}, {obj_id:43902, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165559}, {obj_id:43904, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192380}, {obj_id:43905, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164036}, {obj_id:43906, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198457}, {obj_id:43907, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166694}, {obj_id:43910, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196512}, {obj_id:43911, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165970}, {obj_id:43912, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190172}, {obj_id:43913, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163239}, {obj_id:43916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198420}, {obj_id:43917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167145}, {obj_id:43918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191525}, {obj_id:43919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190153}, {obj_id:43920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162283}, {obj_id:43921, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196379}, {obj_id:43922, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166376}, {obj_id:43923, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183894}, {obj_id:43924, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159068}, {obj_id:43925, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192355}, {obj_id:43926, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164019}, {obj_id:43927, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181986}, {obj_id:43928, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158336}, {obj_id:43929, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155366}, {obj_id:43930, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149136}, {obj_id:43931, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150600}, {obj_id:43933, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194326}, {obj_id:43934, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165167}, {obj_id:43935, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242084}, {obj_id:43937, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239847}, {obj_id:43939, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184120}, {obj_id:43940, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158668}, {obj_id:43941, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188386}, {obj_id:43942, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160694}, {obj_id:43943, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241982}, {obj_id:43945, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192218}, {obj_id:43946, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163510}, {obj_id:43950, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188212}, {obj_id:43951, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161552}, {obj_id:43952, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182092}, {obj_id:43953, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157925}, {obj_id:43955, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190310}, {obj_id:43956, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162328}, {obj_id:43959, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177852}, {obj_id:43960, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156382}, {obj_id:43961, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184029}, {obj_id:43962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159081}, {obj_id:43963, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186068}, {obj_id:43964, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159851}, {obj_id:43966, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237619}, {obj_id:43967, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190235}, {obj_id:43968, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161845}, {obj_id:43969, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229386}, {obj_id:43970, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180013}, {obj_id:43971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157150}, {obj_id:43972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190188}, {obj_id:43973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162766}, {obj_id:43974, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190379}, {obj_id:43975, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162374}, {obj_id:43977, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181994}, {obj_id:43978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158341}, {obj_id:43980, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200528}, {obj_id:43981, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167941}, {obj_id:43982, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194333}, {obj_id:43983, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165174}, {obj_id:43985, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186241}, {obj_id:43986, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160356}, {obj_id:43988, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192305}, {obj_id:43989, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163100}, {obj_id:43991, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239789}, {obj_id:43992, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235419}, {obj_id:43993, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188226}, {obj_id:43994, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161997}, {obj_id:43996, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184079}, {obj_id:43997, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159571}, {obj_id:43999, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190148}, {obj_id:44000, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162726}, {obj_id:44001, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186031}, {obj_id:44002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160766}, {obj_id:44003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190204}, {obj_id:44004, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163198}, {obj_id:44006, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194309}, {obj_id:44007, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165195}, {obj_id:44008, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184162}, {obj_id:44009, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159136}, {obj_id:44011, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190381}, {obj_id:44012, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162376}, {obj_id:44015, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145092}, {obj_id:44016, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177742}, {obj_id:44017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147486}, {obj_id:44019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177586}, {obj_id:44020, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156804}, {obj_id:44021, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196455}, {obj_id:44022, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165513}, {obj_id:44023, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179668}, {obj_id:44024, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157569}, {obj_id:44025, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220754}, {obj_id:44027, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194139}, {obj_id:44028, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164700}, {obj_id:44031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181611}, {obj_id:44032, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158745}, {obj_id:44034, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237685}, {obj_id:44035, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195880}, {obj_id:44036, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233597}, {obj_id:44037, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188196}, {obj_id:44038, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161087}, {obj_id:44040, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177676}, {obj_id:44041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155847}, {obj_id:44042, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181953}, {obj_id:44043, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157841}, {obj_id:44044, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184065}, {obj_id:44045, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159563}, {obj_id:44048, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224959}, {obj_id:44049, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192471}, {obj_id:44050, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163607}, {obj_id:44051, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239682}, {obj_id:44052, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198360}, {obj_id:44053, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167592}, {obj_id:44054, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194534}, {obj_id:44055, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164324}, {obj_id:44056, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194411}, {obj_id:44057, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163857}, {obj_id:44058, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179754}, {obj_id:44059, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157061}, {obj_id:44060, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184038}, {obj_id:44061, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159119}, {obj_id:44062, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163560}, {obj_id:44063, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152093}, {obj_id:44064, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194452}, {obj_id:44065, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164786}, {obj_id:44067, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:44068, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167202}, {obj_id:44069, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186123}, {obj_id:44070, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160350}, {obj_id:44072, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192424}, {obj_id:44073, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162651}, {obj_id:44074, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192224}, {obj_id:44075, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164429}, {obj_id:44076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171127}, {obj_id:44077, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192181}, {obj_id:44078, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163977}, {obj_id:44079, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186007}, {obj_id:44080, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160761}, {obj_id:44081, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181720}, {obj_id:44082, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158256}, {obj_id:44084, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186003}, {obj_id:44085, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160298}, {obj_id:44086, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229187}, {obj_id:44087, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231389}, {obj_id:44088, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188082}, {obj_id:44089, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161992}, {obj_id:44090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136453}, {obj_id:44092, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182098}, {obj_id:44093, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157928}, {obj_id:44094, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188226}, {obj_id:44095, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160646}, {obj_id:44096, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196462}, {obj_id:44097, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165520}, {obj_id:44098, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184269}, {obj_id:44099, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158714}, {obj_id:44100, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:244056}, {obj_id:44102, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192270}, {obj_id:44103, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162638}, {obj_id:44104, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188186}, {obj_id:44105, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161593}, {obj_id:44106, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186031}, {obj_id:44107, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203
[2023-07-23 11:04:39.202320] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2032) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=1573] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:39.202634] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.203532] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EE-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.204447] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.216193] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039b4a628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479053991, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.216315] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479160261 2023-07-23 03:04:39.160261|[process_begin] u=0 in_queue_time:6189512, receive_ts:1690081472970748, enqueue_ts:1690081479160258|[start_sql] u=1 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=66 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=4 |[pc_choose_plan] u=6 |[check_priv] u=6 |[plan_id] u=1 plan_id:45401|[do_open_plan_begin] u=3 plan_id:45401|[sql_start_stmt_begin] u=1 |[post_packet] u=16 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=55895 |[do_open_plan_end] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=4 |[query_end] u=32 |[process_end] u=2 run_ts:1690081479160266|total_timeu=56052)
[2023-07-23 11:04:39.221042] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479138269, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908673}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479164608, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.221097] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] will sleep(sleep_us=86000, remain_us=21455441, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=86, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:39.230022] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.231595] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FA-0-0] [lt=4][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.231607] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FA-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.231615] WDIAG [SHARE.LOCATION] detect_ls_leaders_ (ob_ls_location_service.cpp:845) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FA-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", addr="10.1.129.154:1001", timeout=2000000)
[2023-07-23 11:04:39.231626] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FA-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.231632] WDIAG [SHARE.LOCATION] detect_ls_leaders_ (ob_ls_location_service.cpp:845) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FA-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", addr="10.1.129.155:1001", timeout=2000000)
[2023-07-23 11:04:39.231798] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FA-0-0] [lt=7][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=1193, arg={addr:"192.168.1.1:1001"})
[2023-07-23 11:04:39.231809] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FA-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=1193, arg={addr:"192.168.1.1:1001"})
[2023-07-23 11:04:39.235405] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479138270, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:125, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.235479] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=1] will sleep(sleep_us=100000, remain_us=15934369, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=122, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:39.237471] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B07-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.237486] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B07-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.239684] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479239670]})
[2023-07-23 11:04:39.239711] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479239704]})
[2023-07-23 11:04:39.239726] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.239742] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=14][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.239762] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.239772] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=9][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.239893] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479239885]})
[2023-07-23 11:04:39.239983] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=13][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.239999] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=15][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.240018] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=12][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.240057] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1EA-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.240069] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1EA-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.240053] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=34][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.240086] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=31] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.240119] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=29] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.240134] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B052-0-0] [lt=10] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479240130, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.240477] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C1F-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.241040] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=24] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479241032})
[2023-07-23 11:04:39.241046] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC2B-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.241730] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=14][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.241748] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.241874] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479240951 2023-07-23 03:04:39.240951|[process_begin] u=0 in_queue_time:3287630, receive_ts:1690081475953320, enqueue_ts:1690081479240939|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=62 |[plc_serialize_begin] u=3 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=31 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=4 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=8 |[do_open_plan_end] u=0 |[get_das_id_begin] u=49 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=17 |[calc_partition_location_begin] u=544 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:427|[do_open_plan_begin] u=8 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=51 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=5 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[close_plan_begin] u=27 |[close_das_task_begin] u=1 |[close_das_task_end] u=29 |[start_end_stmt] u=22 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=18 |[process_end] u=4 run_ts:1690081479240954|total_timeu=921)
[2023-07-23 11:04:39.250112] INFO  [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:374) [83349][T1001_TenantMet][T1001][Y0-0000000000000000-0-0] [lt=32] Recycle 0 table(ret=0, allocator_={used:1423565, total:4280115}, tablet_pool_={typeid(T).name():"N9oceanbase7storage8ObTabletE", sizeof(T):2560, used_obj_cnt:728, free_obj_hold_cnt:12, allocator used:1941760, allocator total:2027648}, sstable_pool_={typeid(T).name():"N9oceanbase12blocksstable9ObSSTableE", sizeof(T):1088, used_obj_cnt:1480, free_obj_hold_cnt:1459, allocator used:3385728, allocator total:3532032}, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):3008, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1920, used_obj_cnt:11, free_obj_hold_cnt:722, allocator used:1454272, allocator total:1504384}, tablet count=728, min_minor_cnt=21, pinned_tablet_cnt=0)
[2023-07-23 11:04:39.251190] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196749}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479139319, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.251225] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=34] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196749}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479139319, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.251250] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479239249, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196749}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479139319, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.251297] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479251066 2023-07-23 03:04:39.251066|[process_begin] u=0 in_queue_time:5975826, receive_ts:1690081473275239, enqueue_ts:1690081479251056|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=44 |[check_priv] u=8 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[insert_open] u=1 |[value_start_calc_row] u=2 |[value_after_calc_row] u=6 |[insert_start] u=1 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=8 |[sync_auto_value] u=2 |[close_plan_begin] u=21 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=89 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=2 |[query_end] u=9 |[process_end] u=0 run_ts:1690081479251068|total_timeu=223)
[2023-07-23 11:04:39.251862] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931D-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.252536] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931D-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.252700] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83036][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=32][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.253186] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931D-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.253664] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1947-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.253842] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931D-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.254054] INFO  [STORAGE] runTimerTask (ob_locality_manager.cpp:634) [3296][LocalityReload][T0][Y0-0000000000000000-0-0] [lt=12] runTimer to refresh locality_info(ret=0)
[2023-07-23 11:04:39.254886] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3295][LocltyRefTask][T1][YB420A01819C-0005FDE6257CF52A-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.254909] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3295][LocltyRefTask][T1][YB420A01819C-0005FDE6257CF52A-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.254957] INFO  [SHARE] load_region (ob_locality_table_operator.cpp:373) [3295][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=0] load region(ret=-4122, locality_info={version:0, local_region:"", local_zone:"", local_idc:"", local_zone_type:3, local_zone_status:3, locality_region_array:[], locality_zone_array:[]})
[2023-07-23 11:04:39.256664] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.256678] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.256710] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13794902, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.256728] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.271816] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.271932] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479271218 2023-07-23 03:04:39.271218|[process_begin] u=0 in_queue_time:3247391, receive_ts:1690081476023826, enqueue_ts:1690081479271209|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=4 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=35 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=0 |[check_priv] u=3 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=1 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=22 |[calc_partition_location_begin] u=479 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=0 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[remote_task_completed] u=33 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=54 |[close_das_task_begin] u=1 |[close_das_task_end] u=25 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[query_end] u=8 |[process_end] u=1 run_ts:1690081479271223|total_timeu=712)
[2023-07-23 11:04:39.291482] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.291524] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479291396 2023-07-23 03:04:39.291396|[process_begin] u=0 in_queue_time:4066706, receive_ts:1690081475224690, enqueue_ts:1690081479291388|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=3 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=28 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[parse_job_begin] u=1 |[parse_job_end] u=8 |[remote_task_completed] u=47 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29245}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=11 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=2 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=9 |[process_end] u=1 run_ts:1690081479291400|total_timeu=127)
[2023-07-23 11:04:39.292144] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.292189] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.292194] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13776692, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.292212] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13885377, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.292223] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.292215] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.292248] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] get wrs ts(ls_id={id:1001}, delta=13885139, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.292265] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.292305] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1002}, delta=14214426, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.292321] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.292348] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1003}, delta=14148567, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.292364] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.300720] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [83391][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=13] start do ls ha handler(ls_id_array_=[{id:1}])
[2023-07-23 11:04:39.300784] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4015] sock error detect by epoll(mask=8221, *s={this:0x7fd0cb36d290, fd:552, err:5, last_decode_time_:1690081429215430, last_write_time_:1690081429216521, read_buffer_.get_consume_sz():396, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.300794] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3390][sql_nio12][T0][Y0-0000000000000000-0-0] [lt=14][errcode=-4015] sock error detect by epoll(mask=8221, *s={this:0x7fd4833f8390, fd:563, err:5, last_decode_time_:1690081429223305, last_write_time_:1690081429223442, read_buffer_.get_consume_sz():2097, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.300807] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3222113109, proxy_sessid=720999913969918284, server_id=3, ret=0)
[2023-07-23 11:04:39.300815] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:795) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=7] sock ref clean, do destroy(*s={this:0x7fd0cb36d290, fd:552, err:5, last_decode_time_:1690081429215430, last_write_time_:1690081429216521, read_buffer_.get_consume_sz():396, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.300819] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3390][sql_nio12][T0][Y0-0000000000000000-0-0] [lt=19] kill and revert session(conn.sessid_=3222112279, proxy_sessid=720999913969918284, server_id=3, ret=0)
[2023-07-23 11:04:39.300828] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:795) [3390][sql_nio12][T0][Y0-0000000000000000-0-0] [lt=7] sock ref clean, do destroy(*s={this:0x7fd4833f8390, fd:563, err:5, last_decode_time_:1690081429223305, last_write_time_:1690081429223442, read_buffer_.get_consume_sz():2097, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.300831] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=9] unlock session of tenant(conn.sessid_=3222113109, proxy_sessid=720999913969918284, conn.tenant_id_=1002)
[2023-07-23 11:04:39.300840] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=9] connection close(sessid=3222113109, proxy_sessid=720999913969918284, tenant_id=1002, server_id=3, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:39.300846] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3390][sql_nio12][T0][Y0-0000000000000000-0-0] [lt=10] unlock session of tenant(conn.sessid_=3222112279, proxy_sessid=720999913969918284, conn.tenant_id_=1002)
[2023-07-23 11:04:39.300853] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3390][sql_nio12][T0][Y0-0000000000000000-0-0] [lt=6] connection close(sessid=3222112279, proxy_sessid=720999913969918284, tenant_id=1002, server_id=3, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:39.300862] INFO  [SERVER] run (obmp_disconnect.cpp:78) [83222][T1002_L0_G0][T1002][Y0-0000000000000000-0-0] [lt=11] free session successfully(sessid=3222113109)
[2023-07-23 11:04:39.300873] INFO  [SERVER] run (obmp_disconnect.cpp:78) [83223][T1002_L0_G0][T1002][Y0-0000000000000000-0-0] [lt=14] free session successfully(sessid=3222112279)
[2023-07-23 11:04:39.303740] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73134578892, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:39.305150] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash time detail, (compute_wash_size_time=99, refresh_score_time=816, wash_time=573)
[2023-07-23 11:04:39.307220] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.311728] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196750}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479200368, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.311761] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=33] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196750}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479200368, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.311787] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479252928, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196750}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479200368, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.311826] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479311599 2023-07-23 03:04:39.311599|[process_begin] u=0 in_queue_time:6651463, receive_ts:1690081472660135, enqueue_ts:1690081479311589|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=50 |[check_priv] u=7 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=9 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=8 |[sync_auto_value] u=3 |[close_plan_begin] u=16 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=87 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=2 |[query_end] u=9 |[process_end] u=1 run_ts:1690081479311601|total_timeu=226)
[2023-07-23 11:04:39.312060] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3389][sql_nio11][T0][Y0-0000000000000000-0-0] [lt=13] new mysql sessid created(conn.sessid_=3222113016, support_ssl=false)
[2023-07-23 11:04:39.312083] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3389][sql_nio11][T0][Y0-0000000000000000-0-0] [lt=10] sm conn init succ(conn.sessid_=3222113016, sess.client_addr_="192.168.1.1:43058")
[2023-07-23 11:04:39.312093] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3389][sql_nio11][T0][Y0-0000000000000000-0-0] [lt=6] accept one succ(*s={this:0x7fd433492030, fd:368, err:0, last_decode_time_:0, last_write_time_:1690081479312083, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.312156] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3389][sql_nio11][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd433492030, fd:368, err:5, last_decode_time_:0, last_write_time_:1690081479312083, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.312165] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3389][sql_nio11][T0][Y0-0000000000000000-0-0] [lt=8] kill and revert session(conn.sessid_=3222113016, proxy_sessid=0, server_id=3, ret=0)
[2023-07-23 11:04:39.312170] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:795) [3389][sql_nio11][T0][Y0-0000000000000000-0-0] [lt=4] sock ref clean, do destroy(*s={this:0x7fd433492030, fd:368, err:5, last_decode_time_:0, last_write_time_:1690081479312083, read_buffer_.get_consume_sz():0, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.312177] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3389][sql_nio11][T0][Y0-0000000000000000-0-0] [lt=6] connection close(sessid=3222113016, proxy_sessid=0, tenant_id=0, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:39.312201] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.312683] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.313138] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.313591] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.321740] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.327459] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=22294, clean_start_pos=8200000, clean_num=200000)
[2023-07-23 11:04:39.330096] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.335517] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=17][errcode=-4283]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.335583] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.340342] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=27][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.340325] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479340304]})
[2023-07-23 11:04:39.340354] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.340359] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.340371] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.340366] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479340354]})
[2023-07-23 11:04:39.340377] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=5][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.340557] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=7][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.340567] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.340578] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=7][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.340598] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=19][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.340616] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=18] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.340635] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=16] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.340643] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B053-0-0] [lt=6] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479340641, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.340667] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479340655]})
[2023-07-23 11:04:39.340709] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1EB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.341119] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.341978] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC2C-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.341974] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=34] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479341962})
[2023-07-23 11:04:39.343032] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.343214] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081479341845 2023-07-23 03:04:39.341845|[process_begin] u=0 in_queue_time:3388524, receive_ts:1690081475953320, enqueue_ts:1690081479341830|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=90 |[plc_serialize_begin] u=3 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=6 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=1 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=40 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=3 |[pc_choose_plan] u=3 |[check_priv] u=5 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=3 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=24 |[do_open_plan_end] u=0 |[get_das_id_begin] u=51 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=3 |[do_local_das_task_end] u=31 |[calc_partition_location_begin] u=855 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=13 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[remote_task_completed] u=59 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=2 |[close_plan_begin] u=7 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[close_plan_begin] u=40 |[close_das_task_begin] u=2 |[close_das_task_end] u=38 |[start_end_stmt] u=25 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=23 |[process_end] u=2 run_ts:1690081479341850|total_timeu=1367)
[2023-07-23 11:04:39.354250] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1948-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.356300] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3324][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=1/1, request done=303730706/342333751, request doing=0/0)
[2023-07-23 11:04:39.356718] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.356741] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13894907, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.356752] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.357057] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1EC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.362153] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196751}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479250835, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.362185] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=32] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196751}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479250835, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.362211] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479340297, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196751}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479250835, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.362250] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479362035 2023-07-23 03:04:39.362035|[process_begin] u=0 in_queue_time:6086796, receive_ts:1690081473275239, enqueue_ts:1690081479362019|[start_sql] u=1 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=0 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=42 |[check_priv] u=7 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[insert_start] u=8 |[get_das_id_begin] u=11 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=8 |[sync_auto_value] u=3 |[close_plan_begin] u=18 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=86 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=10 |[process_end] u=1 run_ts:1690081479362038|total_timeu=214)
[2023-07-23 11:04:39.362560] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931E-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.362971] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.363080] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479252927, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908674}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479306614, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.363130] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=1] will sleep(sleep_us=87000, remain_us=21313408, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=87, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:39.363367] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.363750] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.372661] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.372778] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081479372114 2023-07-23 03:04:39.372114|[process_begin] u=0 in_queue_time:3348288, receive_ts:1690081476023826, enqueue_ts:1690081479372097|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=32 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=444 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=6 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=29 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=58 |[close_das_task_begin] u=1 |[close_das_task_end] u=25 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=8 |[process_end] u=1 run_ts:1690081479372117|total_timeu=663)
[2023-07-23 11:04:39.372848] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83283][T1001_LogUpdate][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.373196] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1ED-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.373791] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=22] Cache replace map node details(ret=0, replace_node_count=6, replace_time=13167, replace_start_pos=22100000, replace_num=100000)
[2023-07-23 11:04:39.377585] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479216124, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.377650] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479321690 2023-07-23 03:04:39.321690|[process_begin] u=0 in_queue_time:6350942, receive_ts:1690081472970748, enqueue_ts:1690081479321680|[start_sql] u=0 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=2 |[session] u=0 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=27 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=3 |[check_priv] u=3 |[plan_id] u=0 plan_id:45401|[do_open_plan_begin] u=2 plan_id:45401|[sql_start_stmt_begin] u=0 |[post_packet] u=8 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=55893 |[do_open_plan_end] u=0 |[close_plan_begin] u=3 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=10 |[process_end] u=1 run_ts:1690081479321692|total_timeu=55959)
[2023-07-23 11:04:39.387820] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ======
[2023-07-23 11:04:39.387840] INFO  [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:730) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=12] get_max_applied_scn(scn={val:1}, this={ls_id_:{id:1}, role_:2, proposal_id_:-1, palf_committed_end_lsn_:{lsn:0}, last_check_scn_:{val:18446744073709551615}, max_applied_cb_scn_:{val:1}})
[2023-07-23 11:04:39.387858] INFO  [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1029) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=14] get_min_unreplayed_log_info(lsn={lsn:99688816093}, scn={val:1690081465461187622}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1690081465462273, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:2112, ret_code_:0}}, next_to_submit_lsn_:{lsn:99688816093}, committed_end_lsn_:{lsn:99688816093}, next_to_submit_scn_:{val:1690081465461187622}, base_lsn_:{lsn:0}, base_scn_:{val:1}, iterator_:{iterator_impl:{buf_:0x7fd46f005000, next_round_pread_size:2121728, curr_read_pos:487, curr_read_buf_start_pos:0, curr_read_buf_end_pos:487, log_storage_:{IteratorStorage:{start_lsn:{lsn:99688815606}, end_lsn:{lsn:99688816093}, read_buf:{buf_len_:643072, buf_:0x7fd46f005000}, block_size:67104768, log_storage_:0x7fd48ddfc070}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1690081465461187621}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:399, scn_:{val:1690081465461187621}, data_checksum:2105707313, flag:1}}, init_mode_version:0, accumlate_checksum:2038933450}}}})
[2023-07-23 11:04:39.388690] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=41] succ to get rec scn(*this={this:0x7fd478e04030, ls_id:{id:1}, tenant_id:1001, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1690048801926620928}, last_push_gc_task_ts_:1690048803399922, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:39.388705] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=15] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd483fe6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1690048802126249}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd483fe6080, snapshot_version:{val:1690048802126249}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:39.388725] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:0}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:0}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1})
[2023-07-23 11:04:39.388738] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7fd4867ec1b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7fd4867f1290, table_type:1, memtable_head:244, memtable_tail:245, t3m:0x7fd4ba7e2030, tables:[null, null, null, null, 0x7fd483fd6080, null, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, tx_data_table:0x7fd4867f2650, ls_tablet_svr:0x7fd4867ec190, slice_allocator:0x7fd4867f2690}, min_rec_scn={val:1690067853855506884}, ls_id_={id:1})
[2023-07-23 11:04:39.389261] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=19] get rec log scn(service_type_=0, rec_log_ts={val:1690081456904556569})
[2023-07-23 11:04:39.389269] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:1690061231333212449})
[2023-07-23 11:04:39.389274] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=3] get rec log scn(service_type_=2, rec_log_ts={val:1690075632203527680})
[2023-07-23 11:04:39.389282] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690061231333212449}, checkpoint_scn_in_ls_meta={val:1690061231333212449}, ls_id={id:1}, service_type="TRANS_ID")
[2023-07-23 11:04:39.389301] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1153) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=9][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.1:1001", has_set_deleted:false}, curr_base_lsn={lsn:99113742336}, new_base_lsn={lsn:99113742336}, lsn={lsn:99113742336})
[2023-07-23 11:04:39.389313] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:184) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=12] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:99113742336}, ls->get_ls_id()={id:1})
[2023-07-23 11:04:39.389319] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:191) [83377][T1001_TxCkpt][T1001][Y0-0000000000000000-0-0] [lt=5] succeed to update_clog_checkpoint(ret=0, ls_cnt=1)
[2023-07-23 11:04:39.391350] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479252930, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:126, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.391403] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] will sleep(sleep_us=100000, remain_us=15778445, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=123, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:39.392193] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.392214] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.392240] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.392251] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.392257] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13876653, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.392281] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13985338, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.392273] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.392288] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.392299] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.392301] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=12][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.392311] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.392315] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=3][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.392313] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=31] get wrs ts(ls_id={id:1001}, delta=13985100, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.392327] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.392351] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1002}, delta=14314387, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.392348] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=46][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479340641})
[2023-07-23 11:04:39.392365] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.392366] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479340641}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.392385] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.392387] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1003}, delta=14248528, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.392394] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.392399] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.392399] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479340641}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:39.392421] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=22][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:39.392431] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:39.392435] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:39.392443] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:39.392476] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=7][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.392484] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.392496] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=8][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.392513] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=16][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.392527] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=14] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.392547] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=18] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.392553] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B054-0-0] [lt=4] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479392551, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.392853] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C2E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.394696] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3323][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=1/1, request done=303730833/342934134, request doing=0/0)
[2023-07-23 11:04:39.394775] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3329][RpcIO][T0][Y0-0000000000000000-0-0] [lt=26] [RPC EASY STAT](log_str=conn count=1/1, request done=303730958/343338625, request doing=0/0)
[2023-07-23 11:04:39.395743] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:39.395836] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=40][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479392551})
[2023-07-23 11:04:39.395876] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=40][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479392551}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.395914] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=37][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.395931] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.395943] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.395959] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:39.395970] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:39.395983] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.395997] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:39.396009] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:39.396020] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:39.396031] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:39.396043] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:39.396055] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:39.396071] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:39.396084] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:39.396098] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:39.396110] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:39.396127] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:42, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:39.396146] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=17][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.396164] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=5828527)
[2023-07-23 11:04:39.396210] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541948267909, cache_obj->added_lc()=false, cache_obj->get_object_id()=20497, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:39.396248] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18] [packet retry query](TRACE=begin_ts=1690081479392317 2023-07-23 03:04:39.392317|[process_begin] u=0 in_queue_time:4167627, receive_ts:1690081475224690, enqueue_ts:1690081479392307|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=25 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=95 |[transform_with_outline_begin] u=68 |[transform_with_outline_end] u=0 |[resolve_begin] u=4 |[resolve_end] u=1150 |[transform_begin] u=38 |[optimize_begin] u=1651 |[tl_calc_part_id_end] u=370 |[get_location_cache_begin] u=1 |[get_location_cache_begin] u=402 |[query_end] u=118 |[process_end] u=2 run_ts:1690081479392319|total_timeu=3929)
[2023-07-23 11:04:39.414177] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3327][RpcIO][T0][YB420A01819C-0005FDE62EAFF135-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=1/1, request done=303731150/342559125, request doing=0/0)
[2023-07-23 11:04:39.414621] INFO  [STORAGE.BLKMGR] inspect_bad_block (ob_block_manager.cpp:1460) [3293][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=23] check macro block(block_info={ref_cnt:1, is_free:false, access_time:1689444274125132}, time_interval=637205289486)
[2023-07-23 11:04:39.422838] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196752}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479310838, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.422874] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=36] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196752}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479310838, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.422902] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479391852, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196752}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479310838, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.422946] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479422681 2023-07-23 03:04:39.422681|[process_begin] u=0 in_queue_time:6762545, receive_ts:1690081472660135, enqueue_ts:1690081479422662|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=55 |[check_priv] u=8 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=1 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[insert_start] u=15 |[get_das_id_begin] u=14 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=26 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=96 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=2 |[query_end] u=11 |[process_end] u=1 run_ts:1690081479422684|total_timeu=264)
[2023-07-23 11:04:39.423586] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931F-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.424362] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931F-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.425100] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931F-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.425853] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307931F-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.429739] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83276][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=32][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.430135] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=5][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.441286] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479441273]})
[2023-07-23 11:04:39.441310] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.441312] INFO  [STORAGE.TRANS] operator() (ob_ts_mgr.h:171) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=26] refresh gts functor(ret=-4122, ret="OB_RPC_POST_ERROR", gts_tenant_info={v:1})
[2023-07-23 11:04:39.441315] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.441324] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.441328] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.441341] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479441320]})
[2023-07-23 11:04:39.441470] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=5][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.441495] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=24][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.441514] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=1] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479441504]})
[2023-07-23 11:04:39.441518] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=13][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.441554] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=35][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.441582] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=26] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.441613] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=26] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.441628] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B055-0-0] [lt=10] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479441624, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.441808] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1EE-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.441988] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3C-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.442855] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3632][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=6][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.442869] INFO  [PALF] do_fetch_log_ (log_sliding_window.cpp:1927) [3632][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=0] do_fetch_log_ finished(ret=0, palf_id=1, self="192.168.1.1:1001", dest="10.1.129.154:1001", fetch_log_count=2048, fetch_start_lsn={lsn:24599326915}, prev_lsn={lsn:24599326793}, fetch_start_log_id=52028848, last_slide_log_id=52028847, fetch_log_size=41943040, accepted_mode_pid=2, last_fetch_req_time=1690081479442867, last_fetch_end_lsn={lsn:24641269955}, last_fetch_max_log_id=52030895, last_fetch_committed_end_lsn={lsn:18446744073709551615}, trigger_type=0, this={palf_id:1, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:52028859, max_lsn:24599328379, max_scn:{val:1690081466617213158}}, group_buffer:{log_group_buffer: start_lsn:{lsn:1279418458}, reuse_lsn:{lsn:24599326915}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:24599326793}, last_submit_end_lsn:{lsn:24599326915}, last_submit_log_id:52028847, last_submit_log_pid:2, max_flushed_lsn:{lsn:24599326793}, max_flushed_end_lsn:{lsn:24599326915}, max_flushed_log_pid:2, committed_end_lsn:{lsn:24599326915}, last_slide_log_id:52028847, last_slide_scn:{val:1690081465406512168}, last_slide_lsn:{lsn:24599326793}, last_slide_end_lsn:{lsn:24599326915}, last_slide_log_pid:2, last_slide_log_accum_checksum:3613971437, last_fetch_end_lsn:{lsn:24641269955}, last_fetch_max_log_id:52030895, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd540198530})
[2023-07-23 11:04:39.442936] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=54] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479442929})
[2023-07-23 11:04:39.442942] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC2D-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.443487] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.443590] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479442864 2023-07-23 03:04:39.442864|[process_begin] u=0 in_queue_time:3489544, receive_ts:1690081475953320, enqueue_ts:1690081479442847|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=4 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=49 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=26 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=7 |[do_open_plan_end] u=0 |[get_das_id_begin] u=28 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=15 |[calc_partition_location_begin] u=441 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=6 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=29 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[close_plan_begin] u=22 |[close_das_task_begin] u=0 |[close_das_task_end] u=22 |[start_end_stmt] u=20 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=14 |[process_end] u=0 run_ts:1690081479442869|total_timeu=724)
[2023-07-23 11:04:39.450253] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.454829] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1949-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.456771] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.456791] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=13994952, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.456801] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.460094] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83150][T1002_ArcSrv][T1001][YB420A01819C-0005FDE628578DB5-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.460110] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83150][T1002_ArcSrv][T1001][YB420A01819C-0005FDE628578DB5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.460599] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83150][T1002_ArcSrv][T1001][YB420A01819C-0005FDE628578DB5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.466952] WDIAG [SHARE] refresh (ob_task_define.cpp:382) [3138][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=11][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4283, dropped:131, tid:80219}])
[2023-07-23 11:04:39.469833] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3694][T1_FreInfoReloa][T1][YB420A01819C-0005FDE72A1D929B-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.469853] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3694][T1_FreInfoReloa][T1][YB420A01819C-0005FDE72A1D929B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.470485] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3694][T1_FreInfoReloa][T1][YB420A01819C-0005FDE72A1D929C-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.472248] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=16][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.472262] INFO  [PALF] do_fetch_log_ (log_sliding_window.cpp:1927) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=1] do_fetch_log_ finished(ret=0, palf_id=1, self="192.168.1.1:1001", dest="10.1.129.154:1001", fetch_log_count=2048, fetch_start_lsn={lsn:99688816093}, prev_lsn={lsn:99688815606}, fetch_start_log_id=146980952, last_slide_log_id=146980951, fetch_log_size=41943040, accepted_mode_pid=1, last_fetch_req_time=1690081479472261, last_fetch_end_lsn={lsn:99730759133}, last_fetch_max_log_id=146982999, last_fetch_committed_end_lsn={lsn:18446744073709551615}, trigger_type=0, this={palf_id:1, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:146980983, max_lsn:99688838460, max_scn:{val:1690081466798649234}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:99688816093}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:99688815606}, last_submit_end_lsn:{lsn:99688816093}, last_submit_log_id:146980951, last_submit_log_pid:1, max_flushed_lsn:{lsn:99688815606}, max_flushed_end_lsn:{lsn:99688816093}, max_flushed_log_pid:1, committed_end_lsn:{lsn:99688816093}, last_slide_log_id:146980951, last_slide_scn:{val:1690081465461187621}, last_slide_lsn:{lsn:99688815606}, last_slide_end_lsn:{lsn:99688816093}, last_slide_log_pid:1, last_slide_log_accum_checksum:2038933450, last_fetch_end_lsn:{lsn:99730759133}, last_fetch_max_log_id:146982999, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd48ddf6530})
[2023-07-23 11:04:39.473249] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=26][errcode=-4038]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.473290] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=1] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196753}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479361361, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.473332] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=42] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196753}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479361361, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.473358] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479441454, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196753}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479361361, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.473399] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479473148 2023-07-23 03:04:39.473148|[process_begin] u=0 in_queue_time:6197908, receive_ts:1690081473275239, enqueue_ts:1690081479473134|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=47 |[check_priv] u=8 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=3 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=8 |[insert_start] u=0 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=21 |[sync_auto_value] u=2 |[close_plan_begin] u=16 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=99 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=11 |[process_end] u=0 run_ts:1690081479473151|total_timeu=250)
[2023-07-23 11:04:39.473663] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.473779] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479473115 2023-07-23 03:04:39.473115|[process_begin] u=0 in_queue_time:3449288, receive_ts:1690081476023826, enqueue_ts:1690081479473100|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=31 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=446 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=6 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=29 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=49 |[close_das_task_begin] u=1 |[close_das_task_end] u=33 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=9 |[process_end] u=1 run_ts:1690081479473118|total_timeu=663)
[2023-07-23 11:04:39.474217] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3633][T1_LogUpdater][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.474253] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EF-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.474270] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.474281] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.474303] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.474285] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EF-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.474310] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.474467] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.474479] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=11][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.474497] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=12][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.474522] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=24][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.474543] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=19] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.474566] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=20] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.474577] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B056-0-0] [lt=7] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479474574, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.475284] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EF-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.476190] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1690081479476183})
[2023-07-23 11:04:39.476195] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC2E-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.476259] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EF-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.476352] WDIAG [SQL.ENG] is_in_blacklist (ob_px_util.cpp:3547) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=13][errcode=0] server in blacklist(addr="10.1.129.154:1001", server_start_time=1690081479476237, alive_data.start_service_time_=1686281947823862)
[2023-07-23 11:04:39.476364] WDIAG [SQL.ENG] dispatch_sqcs (ob_dfo_scheduler.cpp:497) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=12][errcode=0] ignore the black server list with virtual table(addr="10.1.129.154:1001", ret=0)
[2023-07-23 11:04:39.476397] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=9][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.476418] INFO  [SQL.ENG] broadcast_dfo (ob_px_interruption.cpp:74) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:3777,from:"192.168.1.1:1001",PX ABORT DFO"}}, addr="192.168.1.1:1001", interrupt_id=task(1686539757600688,12889097246))
[2023-07-23 11:04:39.476429] WDIAG [SQL.ENG] broadcast_px (ob_px_interruption.cpp:42) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=10][errcode=0] fail interrupt dfo(idx=0, ret=0)
[2023-07-23 11:04:39.476437] WDIAG [SQL.ENG] check_all_sqc (ob_px_coord_op.cpp:818) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=5][errcode=0] server not alive(access_locations=[{tablet_id:{id:1}, ls_id:{id:1}, server:"10.1.129.154:1001", need_refresh:0}], sqc->get_access_table_location_keys()=[{table_location_key:-2, ref_table_id:12287, tablet_id:{id:1}, is_dml:false, is_loc_uncertain:false}])
[2023-07-23 11:04:39.476466] INFO  [SQL.ENG] broadcast_dfo (ob_px_interruption.cpp:74) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:3777,from:"192.168.1.1:1001",PX ABORT DFO"}}, addr="192.168.1.1:1001", interrupt_id=task(1686539757600688,12889097246))
[2023-07-23 11:04:39.476474] WDIAG [SQL.ENG] broadcast_px (ob_px_interruption.cpp:42) [3777][BlackListServic][T1][YB420A01819C-0005FDE71EC8AD24-0-0] [lt=7][errcode=0] fail interrupt dfo(idx=0, ret=0)
[2023-07-23 11:04:39.477248] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793EF-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.483306] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.483792] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83127][T1002_LSMetaCh][T1001][YB420A01819C-0005FDE7212A8A56-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.483805] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83127][T1002_LSMetaCh][T1001][YB420A01819C-0005FDE7212A8A56-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.486612] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83364][T1001_LSMetaCh][T1][YB420A01819C-0005FDE7213A8990-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.486628] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83364][T1001_LSMetaCh][T1][YB420A01819C-0005FDE7213A8990-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.489490] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.489636] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3326][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=1/1, request done=303730633/343318436, request doing=1/1)
[2023-07-23 11:04:39.489641] INFO  [CLOG] handle_sync_request (log_request_handler.cpp:486) [83648][T1002_L0_G1][T1002][YB420A01819C-0005FDE7216A81B4-0-0] [lt=13] i am not leader(ret=-4038, palf_id=1003, req={get_type:1}, role=2, is_pending_state=true)
[2023-07-23 11:04:39.489995] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BEF-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.490010] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BEF-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.490024] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C20-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.490031] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C2F-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.490039] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C20-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.490051] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C2F-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.490067] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3334][RpcIO][T0][YB420A01819C-0005FDE72DCBDECF-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=1/1, request done=303731088/342184262, request doing=0/0)
[2023-07-23 11:04:39.490172] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3D-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.490187] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3D-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.490239] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3335][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/1, request done=303731087/342537637, request doing=0/0)
[2023-07-23 11:04:39.490304] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3336][RpcIO][T0][YB420A01819C-0005FDE72E6CB09F-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/1, request done=303730936/343748761, request doing=0/0)
[2023-07-23 11:04:39.490327] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3333][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=1/1, request done=303731071/343732280, request doing=0/0)
[2023-07-23 11:04:39.490403] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3328][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/1, request done=303730839/342494224, request doing=0/0)
[2023-07-23 11:04:39.490516] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3332][RpcIO][T0][YB420A01819C-0005FDE72B4C543D-0-0] [lt=23] [RPC EASY STAT](log_str=conn count=1/1, request done=303730945/343464968, request doing=0/0)
[2023-07-23 11:04:39.490527] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3325][RpcIO][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=1/1, request done=303731159/343565041, request doing=0/0)
[2023-07-23 11:04:39.490568] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3331][RpcIO][T0][YB420A01819C-0005FDE72B4C5444-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=1/1, request done=303731620/342421205, request doing=0/0)
[2023-07-23 11:04:39.490587] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3322][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/1, request done=303731511/343004007, request doing=0/0)
[2023-07-23 11:04:39.490720] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3337][RpcIO][T0][YB420A01819C-0005FDE72EAFC4F7-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=1/1, request done=303731289/343247565, request doing=0/0)
[2023-07-23 11:04:39.490906] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3330][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=1/1, request done=303731293/343280497, request doing=0/0)
[2023-07-23 11:04:39.491471] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:123, local_retry_times:123, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.491493] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=19][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.491502] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.491510] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.491524] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.491585] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.492252] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1}, delta=13976779, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.492271] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.492304] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.492305] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1001}, delta=14085226, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.492319] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.492331] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14085464, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.492340] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1002}, delta=14414513, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.492343] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.492353] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.492373] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1003}, delta=14348654, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.492386] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.493390] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.493432] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479493308 2023-07-23 03:04:39.493308|[process_begin] u=0 in_queue_time:4268617, receive_ts:1690081475224690, enqueue_ts:1690081479493299|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=3 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=26 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=2 |[parse_job_end] u=8 |[remote_task_completed] u=45 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29247}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=11 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=1 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=9 |[process_end] u=1 run_ts:1690081479493312|total_timeu=123)
[2023-07-23 11:04:39.494943] INFO  [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1331) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=9] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001)
[2023-07-23 11:04:39.494955] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1022) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=10] start lib cache evict(tenant_id=1001, mem_hold=98566144, mem_limit=118111600, cache_obj_num=285, cache_node_num=285)
[2023-07-23 11:04:39.494965] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1039) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=9] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=98566144, mem_limit=118111600, cache_obj_num=285, cache_node_num=285)
[2023-07-23 11:04:39.494973] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2024) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=8] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:39.496576] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83344][T1001_FreInfoRe][T1001][YB420A01819C-0005FDE72B8C6368-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.496590] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83344][T1001_FreInfoRe][T1001][YB420A01819C-0005FDE72B8C6368-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.497059] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83344][T1001_FreInfoRe][T1001][YB420A01819C-0005FDE72B8C6369-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.498015] INFO  [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1799) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=5] Dumping All Cache Objs(alloc_obj_list.count()=285, alloc_obj_list=[{obj_id:20, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109179}, {obj_id:154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116796}, {obj_id:195, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106976}, {obj_id:196, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129603}, {obj_id:197, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117342}, {obj_id:200, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115569}, {obj_id:201, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115590}, {obj_id:210, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117618}, {obj_id:213, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110485}, {obj_id:216, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109440}, {obj_id:217, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125156}, {obj_id:226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108544}, {obj_id:227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115837}, {obj_id:229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108552}, {obj_id:232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109728}, {obj_id:234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111259}, {obj_id:385, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88881}, {obj_id:390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:11552}, {obj_id:391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:916091}, {obj_id:392, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106461}, {obj_id:393, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134402}, {obj_id:394, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137472}, {obj_id:395, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132350}, {obj_id:399, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:400, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107490}, {obj_id:401, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110398}, {obj_id:403, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124488}, {obj_id:404, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121664}, {obj_id:409, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175250}, {obj_id:413, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156033}, {obj_id:414, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145796}, {obj_id:421, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144114}, {obj_id:422, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131804}, {obj_id:423, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167182}, {obj_id:425, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236458}, {obj_id:427, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219915}, {obj_id:432, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143476}, {obj_id:433, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164310}, {obj_id:434, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220663}, {obj_id:435, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203920}, {obj_id:439, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153942}, {obj_id:440, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132418}, {obj_id:441, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213169}, {obj_id:442, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176390}, {obj_id:443, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:428668}, {obj_id:444, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208446}, {obj_id:445, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:420917}, {obj_id:449, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143185}, {obj_id:450, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131501}, {obj_id:451, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162857}, {obj_id:452, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:397356}, {obj_id:453, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216472}, {obj_id:454, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:471683}, {obj_id:455, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199451}, {obj_id:458, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145055}, {obj_id:459, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132111}, {obj_id:460, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169934}, {obj_id:462, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249916}, {obj_id:464, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240171}, {obj_id:467, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147492}, {obj_id:468, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182923}, {obj_id:470, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:299640}, {obj_id:472, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286401}, {obj_id:476, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132741}, {obj_id:477, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171395}, {obj_id:478, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:154355}, {obj_id:479, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251338}, {obj_id:480, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182132}, {obj_id:483, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145342}, {obj_id:484, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172824}, {obj_id:485, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135088}, {obj_id:486, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257151}, {obj_id:487, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159098}, {obj_id:488, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242027}, {obj_id:491, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:148679}, {obj_id:492, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175363}, {obj_id:496, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143827}, {obj_id:497, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165757}, {obj_id:499, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226625}, {obj_id:500, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:503, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:479760}, {obj_id:507, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123880}, {obj_id:512, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144720}, {obj_id:513, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132287}, {obj_id:514, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155764}, {obj_id:515, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229546}, {obj_id:519, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140461}, {obj_id:520, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165558}, {obj_id:523, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145927}, {obj_id:1085, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172321}, {obj_id:1088, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:496026}, {obj_id:1089, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:590667}, {obj_id:1108, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:571203}, {obj_id:1534, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111508}, {obj_id:1720, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110232}, {obj_id:1721, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108568}, {obj_id:2813, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:904134}, {obj_id:2814, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1074639}, {obj_id:2824, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126601}, {obj_id:2825, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149042}, {obj_id:2828, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196135}, {obj_id:2829, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225299}, {obj_id:2900, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:382028}, {obj_id:2904, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:504810}, {obj_id:2910, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160620}, {obj_id:2914, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257326}, {obj_id:2927, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159996}, {obj_id:2943, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:320727}, {obj_id:2944, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:398335}, {obj_id:2945, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213512}, {obj_id:2946, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:232858}, {obj_id:2947, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231038}, {obj_id:2948, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:259955}, {obj_id:2955, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215695}, {obj_id:2956, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236436}, {obj_id:2977, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:364316}, {obj_id:2979, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:476386}, {obj_id:2986, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171290}, {obj_id:2987, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173475}, {obj_id:2991, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160324}, {obj_id:2994, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150077}, {obj_id:3004, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1077957}, {obj_id:3018, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193338}, {obj_id:3019, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215348}, {obj_id:3088, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:373210}, {obj_id:3091, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:491688}, {obj_id:3111, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228603}, {obj_id:3183, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:375409}, {obj_id:3187, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:495398}, {obj_id:3192, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114403}, {obj_id:3194, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132114}, {obj_id:3196, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142870}, {obj_id:3197, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161389}, {obj_id:3198, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212913}, {obj_id:3199, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208931}, {obj_id:3200, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245295}, {obj_id:3201, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191638}, {obj_id:3203, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133322}, {obj_id:3206, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143237}, {obj_id:3208, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136758}, {obj_id:3212, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132420}, {obj_id:3215, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133954}, {obj_id:3217, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134278}, {obj_id:3219, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134586}, {obj_id:3224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132714}, {obj_id:3226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130839}, {obj_id:3227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228133}, {obj_id:3229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133030}, {obj_id:3233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221640}, {obj_id:3260, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241140}, {obj_id:3279, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:261670}, {obj_id:3280, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:307076}, {obj_id:3344, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211307}, {obj_id:3346, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229535}, {obj_id:3354, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198169}, {obj_id:3355, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209793}, {obj_id:3364, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371003}, {obj_id:3367, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:488294}, {obj_id:3386, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161936}, {obj_id:3417, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:259444}, {obj_id:3418, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:302716}, {obj_id:3421, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228841}, {obj_id:3423, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272560}, {obj_id:3424, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:323080}, {obj_id:3425, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233184}, {obj_id:3427, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263482}, {obj_id:3429, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:266008}, {obj_id:3430, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313559}, {obj_id:3431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257249}, {obj_id:3432, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:302583}, {obj_id:3435, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209137}, {obj_id:3436, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226426}, {obj_id:3443, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:377617}, {obj_id:3446, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:498848}, {obj_id:3450, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162941}, {obj_id:3452, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120793}, {obj_id:3453, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125150}, {obj_id:3454, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106336}, {obj_id:3455, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166062}, {obj_id:3456, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164083}, {obj_id:3457, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:3458, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176583}, {obj_id:3459, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167827}, {obj_id:3460, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167224}, {obj_id:3461, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250859}, {obj_id:3476, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163876}, {obj_id:3494, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:311929}, {obj_id:3495, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:382142}, {obj_id:3502, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193818}, {obj_id:3503, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202903}, {obj_id:3504, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:305358}, {obj_id:3505, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:373402}, {obj_id:3512, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290500}, {obj_id:3513, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371313}, {obj_id:3526, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237867}, {obj_id:3527, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137034}, {obj_id:3529, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149414}, {obj_id:3530, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155318}, {obj_id:3531, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149088}, {obj_id:3532, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150552}, {obj_id:3533, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158615}, {obj_id:3534, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109997}, {obj_id:3535, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218405}, {obj_id:3573, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204758}, {obj_id:3574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219421}, {obj_id:3579, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202572}, {obj_id:3580, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216139}, {obj_id:3597, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231390}, {obj_id:3640, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241951}, {obj_id:3641, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276520}, {obj_id:3646, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200355}, {obj_id:3647, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212858}, {obj_id:3648, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250703}, {obj_id:3649, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290613}, {obj_id:3650, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:318466}, {obj_id:3651, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:392907}, {obj_id:3691, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234643}, {obj_id:3692, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169830}, {obj_id:3693, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166691}, {obj_id:3694, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176446}, {obj_id:3695, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177696}, {obj_id:3700, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185061}, {obj_id:3701, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189642}, {obj_id:3702, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182888}, {obj_id:3703, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186225}, {obj_id:3704, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206937}, {obj_id:3705, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222717}, {obj_id:3706, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189429}, {obj_id:3707, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174138}, {obj_id:3708, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196108}, {obj_id:3709, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173051}, {obj_id:3710, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196007}, {obj_id:3711, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206080}, {obj_id:3712, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:355688}, {obj_id:3713, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:309741}, {obj_id:3714, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:451053}, {obj_id:3715, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:379695}, {obj_id:3716, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191644}, {obj_id:3717, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199480}, {obj_id:3718, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180699}, {obj_id:3719, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183816}, {obj_id:3720, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178489}, {obj_id:3721, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179453}, {obj_id:3722, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187243}, {obj_id:3723, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192970}, {obj_id:3724, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:307577}, {obj_id:3725, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:376128}, {obj_id:3726, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235392}, {obj_id:3727, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:266803}, {obj_id:3728, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:294863}, {obj_id:3729, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:378201}, {obj_id:3730, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:385496}, {obj_id:3733, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227367}, {obj_id:3735, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:519272}, {obj_id:3738, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264456}, {obj_id:3740, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215149}, {obj_id:3741, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117694}, {obj_id:3742, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211909}, {obj_id:3743, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124622}, {obj_id:3744, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128288}, {obj_id:3745, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110732}, {obj_id:3746, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:122619}, {obj_id:3747, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122641}, {obj_id:3748, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118086}, {obj_id:3749, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110316}, {obj_id:3750, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:121815}, {obj_id:3751, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125255}, {obj_id:3752, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112634}, {obj_id:3753, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119794}, {obj_id:3754, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176518}, {obj_id:3755, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164881}, {obj_id:3756, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113232}, {obj_id:3757, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142629}, {obj_id:3758, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121301}, {obj_id:3759, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112702}, {obj_id:3760, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129293}, {obj_id:3761, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201012}, {obj_id:3762, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137606}, {obj_id:3763, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136771}, {obj_id:3764, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167626}, {obj_id:3766, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163100}, {obj_id:3768, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122303}, {obj_id:3769, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130157}, {obj_id:3770, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172025}, {obj_id:3771, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170852}])
[2023-07-23 11:04:39.498547] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2032) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=531] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:39.506068] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.506080] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081479450245], now0=[mts=1690081479450245], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.506090] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908675}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479449882, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.506117] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=25][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:39.506130] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908675}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479449882, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.506152] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=20] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479441453, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908675}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479449882, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.506188] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=35][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.506196] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.506203] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.506207] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:39.506213] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:39.506221] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=88, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:39.506231] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8] will sleep(sleep_us=88000, remain_us=21170307, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=88, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:39.516307] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.527748] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73134578892, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:39.529245] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=22] Wash time detail, (compute_wash_size_time=112, refresh_score_time=867, wash_time=606)
[2023-07-23 11:04:39.530190] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.533882] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24][errcode=-4038]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.533998] INFO  [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:796) [66390][T1002_PsCacheEv][T1002][Y0-0000000000000000-0-0] [lt=16] ps cache evict(tenant_id=1002, stmt_id_map_.size()=120, stmt_info_map_.size()=120, op.get_used_size()=217964, get_mem_high()=478351980, expired_stmt_ids.count()=0, closed_stmt_ids.count()=120)
[2023-07-23 11:04:39.533961] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196754}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479422443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.534021] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=59] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196754}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479422443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.534071] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=48] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479488815, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196754}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479422443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.534147] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479533686 2023-07-23 03:04:39.533686|[process_begin] u=0 in_queue_time:6873550, receive_ts:1690081472660135, enqueue_ts:1690081479533672|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=2 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=101 |[check_priv] u=15 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=3 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=5 |[exec_plan_begin] u=0 |[exec_plan_end] u=4 |[do_open_plan_end] u=0 |[insert_open] u=3 |[value_start_calc_row] u=5 |[value_after_calc_row] u=12 |[insert_start] u=0 |[get_das_id_begin] u=23 |[get_das_id_end] u=1 |[insert_end] u=4 |[do_local_das_task_begin] u=3 |[do_local_das_task_end] u=37 |[sync_auto_value] u=5 |[close_plan_begin] u=36 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=2 |[end_stmt] u=164 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=5 |[query_end] u=18 |[process_end] u=2 run_ts:1690081479533691|total_timeu=459)
[2023-07-23 11:04:39.534423] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F2-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.534892] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F2-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.535353] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F2-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.535798] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F2-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.538940] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.538951] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081572900748, now=[mts=1690081479483297], now0=[mts=1690081479483297], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.538963] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479377092, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.538998] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=31][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:1003}, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60})
[2023-07-23 11:04:39.539012] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=14][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=3222118872, *tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479377092, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.539036] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=23] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479377092, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.539077] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=41][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:501113, schema_version:1689916332789704, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.539088] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.539095] WDIAG [SQL] open (ob_result_set.cpp:157) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.539105] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:40, local_retry_times:0, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:2, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.539116] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] result set open failed, check if need retry(ret=-4283, cli_ret=-4283, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:39.539128] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] close result set fail(cret=-4283)
[2023-07-23 11:04:39.539136] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=6][errcode=-4283] execute query fail(ret=-4283, timeout_timestamp=1690081572970748)
[2023-07-23 11:04:39.539145] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] try to execute again(ret=-4283, type=1, retry_type=2, timeout_remain=93431604)
[2023-07-23 11:04:39.539162] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8] [packet retry query](TRACE=begin_ts=1690081479483238 2023-07-23 03:04:39.483238|[process_begin] u=0 in_queue_time:6512490, receive_ts:1690081472970748, enqueue_ts:1690081479483228|[start_sql] u=1 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=33 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=3 |[check_priv] u=4 |[plan_id] u=1 plan_id:45401|[do_open_plan_begin] u=1 plan_id:45401|[sql_start_stmt_begin] u=1 |[post_packet] u=16 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=55783 |[do_open_plan_end] u=7 |[close_plan_begin] u=29 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=32 |[process_end] u=1 run_ts:1690081479483241|total_timeu=55922)
[2023-07-23 11:04:39.541897] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.541929] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479541889]})
[2023-07-23 11:04:39.541957] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479541947]})
[2023-07-23 11:04:39.541959] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.541971] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=11][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.541992] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.542001] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=9][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.542167] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=13][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.542186] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=18][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.542205] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=12][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.542238] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=32][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.542261] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479542248]})
[2023-07-23 11:04:39.542267] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=28] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.542310] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=39] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.542324] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B057-0-0] [lt=10] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479542321, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.542440] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1EF-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.542718] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF0-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.543816] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=32] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479543810})
[2023-07-23 11:04:39.543822] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC2F-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.543833] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3291][][T0][YB420A01819C-0005FDE715FDDC2F-0-0] [lt=9][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.544376] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.544474] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479543745 2023-07-23 03:04:39.543745|[process_begin] u=0 in_queue_time:3590424, receive_ts:1690081475953320, enqueue_ts:1690081479543738|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=51 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=28 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[exec_plan_end] u=7 |[do_open_plan_end] u=0 |[get_das_id_begin] u=30 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=15 |[calc_partition_location_begin] u=439 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=7 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=36 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=23 |[close_das_task_begin] u=1 |[close_das_task_end] u=21 |[start_end_stmt] u=15 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=15 |[process_end] u=1 run_ts:1690081479543747|total_timeu=728)
[2023-07-23 11:04:39.547354] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.547365] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081479491577], now0=[mts=1690081479491577], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.547379] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=14][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:127, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.547406] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=25][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:39.547417] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:127, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.547439] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=21] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479488814, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:127, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.547477] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=38][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.547487] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.547492] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.547498] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:39.547502] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:39.547510] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=124, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:39.547519] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9] will sleep(sleep_us=100000, remain_us=15622329, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=124, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:39.552610] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=23350, clean_start_pos=8400000, clean_num=200000)
[2023-07-23 11:04:39.555386] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194A-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.555401] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194A-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.556825] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.556843] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14095008, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.556851] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.556864] INFO  [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [WRS] update tenant weak read server version(tenant_id=1001, server_version={version:{val:1690081465461187621}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0, epoch_tstamp:1690081479556835}, version_delta=14095676)
[2023-07-23 11:04:39.575018] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.575205] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479573992 2023-07-23 03:04:39.573992|[process_begin] u=0 in_queue_time:3550165, receive_ts:1690081476023826, enqueue_ts:1690081479573976|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=4 |[session] u=2 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=55 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=5 |[tl_calc_part_id_end] u=5 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=3 |[pc_choose_plan] u=1 |[check_priv] u=4 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=4 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[get_das_id_begin] u=6 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=27 |[calc_partition_location_begin] u=847 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=13 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[remote_task_completed] u=53 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=3 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=8 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=1 |[close_plan_begin] u=83 |[close_das_task_begin] u=1 |[close_das_task_end] u=42 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=15 |[process_end] u=1 run_ts:1690081479573997|total_timeu=1211)
[2023-07-23 11:04:39.579490] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B83-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.579505] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B83-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.584253] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196755}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479473022, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.584290] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=36] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196755}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479473022, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.584323] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=32] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479541446, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196755}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479473022, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.584363] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479584112 2023-07-23 03:04:39.584112|[process_begin] u=0 in_queue_time:6308873, receive_ts:1690081473275239, enqueue_ts:1690081479584099|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=5 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=48 |[check_priv] u=9 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=1 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[insert_start] u=11 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=20 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=99 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=10 |[process_end] u=1 run_ts:1690081479584118|total_timeu=250)
[2023-07-23 11:04:39.584716] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079320-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.584731] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079320-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.585191] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079320-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.585642] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079320-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.586086] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079320-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.588584] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] Cache replace map node details(ret=0, replace_node_count=0, replace_time=14712, replace_start_pos=22200000, replace_num=100000)
[2023-07-23 11:04:39.592211] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=89][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.592226] INFO  [PALF] do_fetch_log_ (log_sliding_window.cpp:1927) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=0] do_fetch_log_ finished(ret=0, palf_id=1001, self="192.168.1.1:1001", dest="10.1.129.154:1001", fetch_log_count=2048, fetch_start_lsn={lsn:704197620496}, prev_lsn={lsn:704197620374}, fetch_start_log_id=406126644, last_slide_log_id=406126643, fetch_log_size=41943040, accepted_mode_pid=1, last_fetch_req_time=1690081479592224, last_fetch_end_lsn={lsn:704239563536}, last_fetch_max_log_id=406128691, last_fetch_committed_end_lsn={lsn:18446744073709551615}, trigger_type=0, this={palf_id:1001, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:406126649, max_lsn:704197632397, max_scn:{val:1690081466012354861}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:704197624828}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:704197620496}, last_submit_end_lsn:{lsn:704197624828}, last_submit_log_id:406126644, last_submit_log_pid:1, max_flushed_lsn:{lsn:704197620496}, max_flushed_end_lsn:{lsn:704197624828}, max_flushed_log_pid:1, committed_end_lsn:{lsn:704197620496}, last_slide_log_id:406126643, last_slide_scn:{val:1690081465406750436}, last_slide_lsn:{lsn:704197620374}, last_slide_end_lsn:{lsn:704197620496}, last_slide_log_pid:1, last_slide_log_accum_checksum:2486464045, last_fetch_end_lsn:{lsn:704239563536}, last_fetch_max_log_id:406128691, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd4653b0530})
[2023-07-23 11:04:39.592320] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.592354] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.592358] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=14076801, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.592374] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.592385] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14185486, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.592395] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.592401] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1001}, delta=14185248, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.592413] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.592435] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get wrs ts(ls_id={id:1002}, delta=14514535, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.592449] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.592470] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1003}, delta=14448676, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.592484] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.594245] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=103][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479542321})
[2023-07-23 11:04:39.594280] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:88, local_retry_times:88, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.594301] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=17][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.594278] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=34][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479542321}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.594308] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.594312] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.594312] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=33][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.594320] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.594326] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.594364] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.594336] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479542321}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:39.594373] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.594376] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=39][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:39.594393] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=17][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:39.594403] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:39.594412] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:39.597109] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:39.597183] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=31][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479542321})
[2023-07-23 11:04:39.597218] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=36][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479542321}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.597251] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=31][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.597264] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.597274] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.597286] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:39.597306] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=19][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:39.597317] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.597328] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:39.597335] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:39.597343] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:39.597352] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:39.597362] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:39.597373] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:39.597386] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:39.597396] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:39.597408] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:39.597417] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:39.597431] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:44, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:39.597446] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.597460] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=5627231)
[2023-07-23 11:04:39.597496] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541948469197, cache_obj->added_lc()=false, cache_obj->get_object_id()=20498, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:39.597527] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14] [packet retry query](TRACE=begin_ts=1690081479594181 2023-07-23 03:04:39.594181|[process_begin] u=0 in_queue_time:4369491, receive_ts:1690081475224690, enqueue_ts:1690081479594170|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=51 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[pc_choose_plan] u=167 |[transform_with_outline_begin] u=100 |[transform_with_outline_end] u=2 |[resolve_begin] u=4 |[resolve_end] u=919 |[transform_begin] u=33 |[optimize_begin] u=1318 |[tl_calc_part_id_end] u=307 |[get_location_cache_begin] u=1 |[get_location_cache_begin] u=337 |[query_end] u=94 |[process_end] u=1 run_ts:1690081479594185|total_timeu=3344)
[2023-07-23 11:04:39.608215] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [3734][T1_LSArchiveMgr][T1][YB420A01819C-0005FDE623878DB5-0-0] [lt=14] gc stale ls task succ
[2023-07-23 11:04:39.617753] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD6-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.617766] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD6-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.630196] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.642919] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479642901]})
[2023-07-23 11:04:39.642943] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=1] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479642936]})
[2023-07-23 11:04:39.642959] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.642976] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=16][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.642998] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.643009] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.643166] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479643156]})
[2023-07-23 11:04:39.643173] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=14][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.643188] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=13][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.643209] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=13][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.643256] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=46][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.643290] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F0-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.643289] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=32] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.643334] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=41] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.643350] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B058-0-0] [lt=12] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479643346, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.643631] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C21-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.644700] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.644743] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC30-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.644740] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=37] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479644729})
[2023-07-23 11:04:39.644731] INFO  [PALF] do_fetch_log_ (log_sliding_window.cpp:1927) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=0] do_fetch_log_ finished(ret=0, palf_id=1, self="192.168.1.1:1001", dest="10.1.129.154:1001", fetch_log_count=2048, fetch_start_lsn={lsn:8392989598}, prev_lsn={lsn:8392989476}, fetch_start_log_id=32949108, last_slide_log_id=32949107, fetch_log_size=41943040, accepted_mode_pid=1, last_fetch_req_time=1690081479644729, last_fetch_end_lsn={lsn:8434932638}, last_fetch_max_log_id=32951155, last_fetch_committed_end_lsn={lsn:18446744073709551615}, trigger_type=0, this={palf_id:1, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:32949107, max_lsn:8392989598, max_scn:{val:1690081465515197326}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:8392989598}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:8392989476}, last_submit_end_lsn:{lsn:8392989598}, last_submit_log_id:32949107, last_submit_log_pid:1, max_flushed_lsn:{lsn:8392989476}, max_flushed_end_lsn:{lsn:8392989598}, max_flushed_log_pid:1, committed_end_lsn:{lsn:8392989598}, last_slide_log_id:32949107, last_slide_scn:{val:1690081465515197326}, last_slide_lsn:{lsn:8392989476}, last_slide_end_lsn:{lsn:8392989598}, last_slide_log_pid:1, last_slide_log_accum_checksum:383597832, last_fetch_end_lsn:{lsn:8434932638}, last_fetch_max_log_id:32951155, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd501ff2530})
[2023-07-23 11:04:39.644788] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=1] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196756}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479534067, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.644822] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=34] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196756}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479534067, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.644850] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479642530, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196756}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479534067, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.644889] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479644642 2023-07-23 03:04:39.644642|[process_begin] u=0 in_queue_time:6984507, receive_ts:1690081472660135, enqueue_ts:1690081479644633|[start_sql] u=1 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=54 |[check_priv] u=8 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=2 |[value_after_calc_row] u=13 |[insert_start] u=1 |[get_das_id_begin] u=12 |[get_das_id_end] u=1 |[insert_end] u=2 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=18 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=91 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=10 |[process_end] u=0 run_ts:1690081479644645|total_timeu=246)
[2023-07-23 11:04:39.645242] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F3-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.645257] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F3-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.645729] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F3-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.645787] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.645967] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081479644616 2023-07-23 03:04:39.644616|[process_begin] u=0 in_queue_time:3691295, receive_ts:1690081475953320, enqueue_ts:1690081479644603|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=2 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=7 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=83 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=5 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=46 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=4 |[pc_choose_plan] u=2 |[check_priv] u=5 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=3 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[exec_plan_end] u=10 |[do_open_plan_end] u=0 |[get_das_id_begin] u=46 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=3 |[do_local_das_task_end] u=24 |[calc_partition_location_begin] u=858 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=12 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[remote_task_completed] u=54 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=3 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=2 |[close_plan_begin] u=7 |[start_end_stmt] u=1 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[close_plan_begin] u=41 |[close_das_task_begin] u=2 |[close_das_task_end] u=40 |[start_end_stmt] u=26 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=24 |[process_end] u=2 run_ts:1690081479644625|total_timeu=1349)
[2023-07-23 11:04:39.646193] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F3-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.646570] INFO  [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [3784][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=25] current monitor number(seq_=-1)
[2023-07-23 11:04:39.646651] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F3-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.647582] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:124, local_retry_times:124, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.647594] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.647598] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.647603] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.647611] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.647648] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.649964] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.649975] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081479594359], now0=[mts=1690081479594359], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.649985] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908676}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479594110, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.650010] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=22][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:39.650022] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908676}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479594110, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.650042] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=19] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479541448, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908676}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479594110, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.650075] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=33][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.650085] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.650090] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.650095] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:39.650099] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:39.650107] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=89, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:39.650116] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9] will sleep(sleep_us=89000, remain_us=21026422, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=89, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:39.655948] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.656880] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.656901] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14195043, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.656909] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.667224] WDIAG [SHARE] refresh (ob_task_define.cpp:382) [3138][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=13][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4038, dropped:370, tid:80219}, {errcode:-4122, dropped:4867, tid:45604}])
[2023-07-23 11:04:39.667493] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1285) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS](tenant_id=500, io_status=group_id: 0, group_name: OTHER_GROUPS, mode:  read, size: 2097152.00, iops:     2.00, rt:  3828.00)
[2023-07-23 11:04:39.667505] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1307) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS](tenant_id=500, ref_cnt=2, io_config={group_num_:0, memory_limit_:536870912, callback_thread_count_:8, unit_config_:{min_iops:10000, max_iops:50000, weight:10000}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=37693728, callback_queues=[0, 0, 0, 0, 0, 0, 0, 0])
[2023-07-23 11:04:39.667528] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=19] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667538] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667545] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667552] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667556] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667566] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667571] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667577] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667582] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667588] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667594] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667600] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667605] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667612] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667619] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.667625] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:39.675773] WDIAG [RPC.FRAME] create_session (ob_req_transport.cpp:263) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=86][errcode=-4122] address in blacklist(ret=-4122, addr="10.1.129.154:1001")
[2023-07-23 11:04:39.675793] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=21][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.675805] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=12][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.675816] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=10][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1, svr="10.1.129.154:1001", timeout=9999948, timeout_timestamp=1690081489675708)
[2023-07-23 11:04:39.675833] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=15] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.675848] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=13][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:39.675859] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:39.675868] WDIAG [SQL] open (ob_result_set.cpp:157) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:39.675878] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:39.675887] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"})
[2023-07-23 11:04:39.675904] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=15][errcode=-4122] execute failed(ret=-4122, tenant_id=1, executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:39.675919] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=14][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:39.675938] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=14][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.675947] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.675955] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.676012] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.676024] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=12][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:39.676036] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=10][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1, sql=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:39.676047] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=11][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1)
[2023-07-23 11:04:39.676057] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1)
[2023-07-23 11:04:39.676071] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=12][errcode=-4122] query failed(ret=-4122, conn=0x7fd1e8d44050, start=1690081479675698, sql=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:39.676084] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=13][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:39.676094] WDIAG [SERVER] next_tenant_ (ob_iterate_private_virtual_table.cpp:255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] execute sql failed(ret=-4122, ret="OB_RPC_POST_ERROR", exec_tenant_id=1, cur_tenant_id=1, sql_=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:39.676107] WDIAG [SERVER] do_open (ob_iterate_private_virtual_table.cpp:138) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=13][errcode=-4122] switch to next tenant failed(ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:39.676116] WDIAG [SERVER] inner_get_next_row (ob_iterate_private_virtual_table.cpp:271) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] do open failed(ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:39.676126] WDIAG [COMMON] get_next_row (ob_virtual_table_iterator.cpp:372) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] fail to inner get next row(ret=-4122, scan_param_={tablet_id:{id:1}, ls_id:{id:1}, column_ids:[16, 20, 23, 27, 22, 26], index_id:12291, key_ranges:[{"range":"table_id:12291,group_idx:0,(MIN;MAX)"}], ss_key_ranges:[{"range":"table_id:12291,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1690081485923826, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup:0, reserved:0}, sql_mode:12582912, reserved_cell_count:6, schema_version:1686281724066464, query_begin_schema_version:1686562084609584, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[{type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:0, res_buf_off:416, dyn_buf_header_offset:400, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT UNSIGNED", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:16, res_buf_off:424, dyn_buf_header_offset:408, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484110}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT UNSIGNED", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:32, res_buf_off:432, dyn_buf_header_offset:416, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd5154841c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:48, res_buf_off:440, dyn_buf_header_offset:424, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484270}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:64, res_buf_off:448, dyn_buf_header_offset:432, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484320}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:80, res_buf_off:456, dyn_buf_header_offset:440, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd5154843d0}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7fd26c606ce8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, table_param:{table_id:12291, output_projector:[0, 1, 2, 3, 4, 5], aggregate_projector:[], output_sel_mask:[true, true, true, true, true, true], pad_col_projector:[], main_read_info:{schema_column_count:13, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:13, is_oracle_mode:false, cols_index:[0, 6, 9, 13, 8, 12], memtable_cols_index:[0, 4, 7, 11, 6, 10], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:8, allocator:0x7fcfd79c2530, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:23, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:26, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fcfd79c2530, is_inited:true}, cols_param:null, index_read_info:NULL}}, use_lob_locator:false, rowid_version:0, rowid_projector:[], enable_lob_locator_v2:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0}, need_scn:false, need_switch_param:false})
[2023-07-23 11:04:39.676412] WDIAG [COMMON] get_next_row (ob_virtual_table_iterator.cpp:472) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=285][errcode=-4122] get next row failed(ret=-4122)
[2023-07-23 11:04:39.676420] WDIAG [SQL.ENG] get_next_row_with_das (ob_table_scan_op.cpp:1683) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] get next row from das result failed(ret=-4122)
[2023-07-23 11:04:39.676430] WDIAG [SQL.ENG] inner_get_next_row_for_tsc (ob_table_scan_op.cpp:1870) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=10][errcode=-4122] fail to get next row from ObNewRowIterator(ret=-4122)
[2023-07-23 11:04:39.676436] WDIAG [SQL.ENG] inner_get_next_row_implement (ob_table_scan_op.cpp:1847) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] failed to get next row(ret=-4122)
[2023-07-23 11:04:39.676445] WDIAG [SQL.ENG] inner_get_next_row (ob_table_scan_op.cpp:2827) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] get next row failed(ret=-4122)
[2023-07-23 11:04:39.676456] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=10][errcode=-4122] inner get next row failed(ret=-4122, type=3, op="PHY_TABLE_SCAN", op_id=0)
[2023-07-23 11:04:39.676468] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=11][errcode=-4122] get next row from operator failed(ret=-4122)
[2023-07-23 11:04:39.676476] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] get next row from exec result failed(ret=-4122)
[2023-07-23 11:04:39.676486] WDIAG [SERVER] response_query_result (ob_query_driver.cpp:233) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] fail to iterate and response(ret=-4122, row_num=0, can_retry=true)
[2023-07-23 11:04:39.676502] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:101) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=15][errcode=-4122] response query result fail(ret=-4122)
[2023-07-23 11:04:39.676512] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:39, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:2, client_ret:-4122}, need_retry=true)
[2023-07-23 11:04:39.676525] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:114) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=12][errcode=-4122] result response failed, check if need retry(ret=-4122, cli_ret=-4122, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:39.676575] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:121) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] close result set fail(cret=-4122)
[2023-07-23 11:04:39.676585] WDIAG [SERVER] do_process (obmp_query.cpp:846) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] execute query fail(ret=-4122, timeout_timestamp=1690081486023826)
[2023-07-23 11:04:39.676597] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] try to execute again(ret=-4122, type=1, retry_type=2, timeout_remain=6347229)
[2023-07-23 11:04:39.676619] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=10] [packet retry query](TRACE=begin_ts=1690081479674889 2023-07-23 03:04:39.674889|[process_begin] u=0 in_queue_time:3651062, receive_ts:1690081476023826, enqueue_ts:1690081479674877|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=47 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=5 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=3 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=5 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=31 |[calc_partition_location_begin] u=726 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=11 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[remote_task_completed] u=94 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=11 |[do_open_plan_end] u=10 |[get_location_cache_begin] u=9 |[close_plan_begin] u=58 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[close_plan_begin] u=596 |[close_das_task_begin] u=1 |[close_das_task_end] u=38 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=42 |[process_end] u=1 run_ts:1690081479674893|total_timeu=1728)
[2023-07-23 11:04:39.686124] INFO  [STORAGE] operator() (ob_tenant_freezer.cpp:125) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=7] ====== tenant freeze timer task ======
[2023-07-23 11:04:39.687040] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=21][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.687060] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=20][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.687075] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=12][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=29999915, timeout_timestamp=1690081509686942)
[2023-07-23 11:04:39.687094] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=16] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.687112] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=16][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:39.687124] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=11][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:39.687134] WDIAG [SQL] open (ob_result_set.cpp:157) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=10][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:39.687150] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=13][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:39.687162] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=9][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT * FROM __all_freeze_info ORDER BY frozen_scn DESC LIMIT 1"})
[2023-07-23 11:04:39.687177] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=14][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT * FROM __all_freeze_info ORDER BY frozen_scn DESC LIMIT 1"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:39.687194] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=16][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:39.687216] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=16][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.687226] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=10][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.687236] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=8][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.687253] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=10][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT * FROM __all_freeze_info ORDER BY frozen_scn DESC LIMIT 1"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.687266] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [83053][T1002_Occam][T1002][YB420A01819C-0005FDE720491AEE-0-0] [lt=12][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT * FROM __all_freeze_info ORDER BY frozen_scn DESC LIMIT 1"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:39.687280] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT * FROM __all_freeze_info ORDER BY frozen_scn DESC LIMIT 1)
[2023-07-23 11:04:39.687300] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:39.687313] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:39.687326] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4122] query failed(ret=-4122, conn=0x7fd54a7f0050, start=1690081479686941, sql=SELECT * FROM __all_freeze_info ORDER BY frozen_scn DESC LIMIT 1)
[2023-07-23 11:04:39.687340] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:39.687351] WDIAG [SHARE] get_freeze_info (ob_freeze_info_proxy.cpp:69) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", sql=SELECT * FROM __all_freeze_info ORDER BY frozen_scn DESC LIMIT 1, tenant_id=1002)
[2023-07-23 11:04:39.687423] WDIAG [STORAGE] get_global_frozen_scn_ (ob_tenant_freezer.cpp:1115) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] get_frozen_scn failed(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002)
[2023-07-23 11:04:39.687435] WDIAG [STORAGE] do_major_if_need_ (ob_tenant_freezer.cpp:1217) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4122] fail to get global frozen version(ret=-4122)
[2023-07-23 11:04:39.687446] WDIAG [STORAGE] check_and_freeze_normal_data_ (ob_tenant_freezer.cpp:408) [83053][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=0] [TenantFreezer] fail to do major freeze(tmp_ret=-4122)
[2023-07-23 11:04:39.692371] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.692384] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.692393] WDIAG [STORAGE.TRANS] post_cluster_heartbeat_rpc_ (ob_tenant_weak_read_service.cpp:803) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4122] post cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", cluster_service_master="10.1.129.154:1001", tenant_id_=1002, req={req_server:"192.168.1.1:1001", version:{val:1690081465077463222}, valid_part_count:4, total_part_count:4, generate_timestamp:1690081479692361}, cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.692405] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.692414] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.692416] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.692419] WDIAG [STORAGE.TRANS] post_cluster_heartbeat_rpc_ (ob_tenant_weak_read_service.cpp:803) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5][errcode=-4122] post cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", cluster_service_master="10.1.129.154:1001", tenant_id_=1, req={req_server:"192.168.1.1:1001", version:{val:1690081465406512168}, valid_part_count:1, total_part_count:1, generate_timestamp:1690081479692398}, cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.692435] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.692432] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:759) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4122] post cluster heartbeat rpc fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1002, local_server_version={val:1690081465077463222}, valid_part_count=4, total_part_count=4, generate_timestamp=1690081479692361)
[2023-07-23 11:04:39.692445] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:759) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4122] post cluster heartbeat rpc fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1, local_server_version={val:1690081465406512168}, valid_part_count=1, total_part_count=1, generate_timestamp=1690081479692398)
[2023-07-23 11:04:39.692445] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:769) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4122] tenant weak read service do cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1002, last_post_cluster_heartbeat_tstamp_=1690081479592343, cluster_heartbeat_interval_=1000000, cluster_service_tablet_id={id:226}, cluster_service_master="10.1.129.154:1001")
[2023-07-23 11:04:39.692454] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:769) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] tenant weak read service do cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1, last_post_cluster_heartbeat_tstamp_=1690081479592371, cluster_heartbeat_interval_=1000000, cluster_service_tablet_id={id:226}, cluster_service_master="10.1.129.154:1001")
[2023-07-23 11:04:39.692460] INFO  [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:808) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=false, can_update_version=false, cur_leader_epoch=0, start_service_tstamp_=0, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0)
[2023-07-23 11:04:39.692472] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1}, delta=14285508, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.692482] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.692490] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1}, delta=14176823, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.692506] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.692530] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] get wrs ts(ls_id={id:1001}, delta=14285270, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.692543] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.692563] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1002}, delta=14614557, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.692576] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.692596] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1003}, delta=14548698, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.692609] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.695169] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=68][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.695183] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.695191] WDIAG [SQL.EXE] task_execute (ob_executor_rpc_impl.cpp:80) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1, svr="10.1.129.154:1001", timeout=5529551, timeout_timestamp=1690081485224690)
[2023-07-23 11:04:39.695194] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:39.695203] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.695217] WDIAG [SQL.EXE] execute (ob_remote_task_executor.cpp:108) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4122] fail post task(ret=-4122)
[2023-07-23 11:04:39.695203] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196757}, scn:1690081479642533}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196757}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479583587, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.695228] WDIAG [SQL.EXE] execute (ob_remote_job_executor.cpp:55) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4122] fail execute task(ret=-4122, *task_info={task_loc:{server:"10.1.129.154:1001", ob_task_id:{ob_job_id:{ob_execution_id:{server:"192.168.1.1:1001", execution_id:18446744073709551615, task_type:0, hash:4219408603045787357}, job_id:29249}, task_id:0, task_cnt:0}}, range_location:{part_locs:[], server:"0.0.0.0:0"}, location_idx:0, location_idx_list:[0], state:5, slice_count_pos:[], background:false, retry_times:0, location_idx_list:[0]})
[2023-07-23 11:04:39.695254] WDIAG [SQL.EXE] execute_with_plan (ob_remote_scheduler.cpp:124) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=26][errcode=-4122] fail execute remote job(ret=-4122)
[2023-07-23 11:04:39.695272] WDIAG [SQL.EXE] execute_with_plan (ob_remote_scheduler.cpp:141) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4122] fail to schedule, print job's status(ret=-4122, print_ret=0, job_status="{job_count:2, {task_count:0, state statistics:{}}, {task_count:1, task_info:{loc:"10.1.129.154:1001"ctrl:"192.168.1.1:1001"eid:18446744073709551615jid:29249tid:0pull_sid:0state:5}state statistics:{failed:1}}}")
[2023-07-23 11:04:39.695284] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:57) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4122] execute with plan failed(ret=-4122)
[2023-07-23 11:04:39.695300] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:39.695306] WDIAG [SQL] open (ob_result_set.cpp:157) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:39.695314] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:45, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:2, client_ret:-4122}, need_retry=true)
[2023-07-23 11:04:39.695328] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4122] result set open failed, check if need retry(ret=-4122, cli_ret=-4122, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:39.695341] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4122] close result set fail(cret=-4122)
[2023-07-23 11:04:39.695350] WDIAG [SERVER] do_process (obmp_query.cpp:846) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4122] execute query fail(ret=-4122, timeout_timestamp=1690081485224690)
[2023-07-23 11:04:39.695364] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4122] try to execute again(ret=-4122, type=1, retry_type=2, timeout_remain=5529327)
[2023-07-23 11:04:39.695231] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196757}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479583587, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.695397] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=164][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:39.695405] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.695411] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:39.695420] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=10][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.695425] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:39.695386] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10] [packet retry query](TRACE=begin_ts=1690081479695067 2023-07-23 03:04:39.695067|[process_begin] u=0 in_queue_time:4470377, receive_ts:1690081475224690, enqueue_ts:1690081479695057|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=38 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=2 |[check_priv] u=3 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=3 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=3 |[parse_job_end] u=5 |[remote_task_completed] u=94 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29249}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=73 |[do_open_plan_end] u=6 |[get_location_cache_begin] u=7 |[close_plan_begin] u=23 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=44 |[process_end] u=1 run_ts:1690081479695071|total_timeu=318)
[2023-07-23 11:04:39.695429] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:39.695433] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:39.695438] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503985, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196757}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479583587, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196757}, scn:1690081479642533}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:39.695842] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=404][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:39.695847] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:39.695857] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:39.695861] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:39.695864] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:39.695870] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:39.695878] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:39.695882] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:39.695887] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:39.695891] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:39.695895] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:39.695899] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:39.695923] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:61, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:39.695937] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196757}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479583587, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.695967] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=30] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196757}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479583587, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.695992] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479642533, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196757}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479583587, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.696021] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:39.696026] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:39.696031] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:39.696038] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=93579202)
[2023-07-23 11:04:39.696053] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081479695098 2023-07-23 03:04:39.695098|[process_begin] u=0 in_queue_time:6419858, receive_ts:1690081473275239, enqueue_ts:1690081479695087|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=5 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=45 |[check_priv] u=7 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=1 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=2 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=11 |[get_das_id_end] u=0 |[insert_end] u=2 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=247 |[sync_auto_value] u=432 |[close_plan_begin] u=63 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=83 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=2 |[query_end] u=31 |[process_end] u=1 run_ts:1690081479695100|total_timeu=954)
[2023-07-23 11:04:39.696737] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=0][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.696756] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=20][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.696768] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999207, timeout_timestamp=1690081480695932)
[2023-07-23 11:04:39.696788] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=17] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.696804] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=15][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:39.696812] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:39.696819] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:39.696836] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:39.696845] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"})
[2023-07-23 11:04:39.696859] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:39.696875] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=15][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:39.696896] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=16][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.696906] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.696912] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.696928] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.696942] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:39.696955] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:39.696973] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=17][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:39.696987] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:39.697000] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=12][errcode=-4122] query failed(ret=-4122, conn=0x7fd54c1e8050, start=1690081479696653, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:39.697010] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:39.697021] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:39.697066] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=12][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:200282})
[2023-07-23 11:04:39.697079] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:200282}, add_timestamp:1690081479695903})
[2023-07-23 11:04:39.697865] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.697880] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=15][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.697890] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999238, timeout_timestamp=1690081480697093)
[2023-07-23 11:04:39.697907] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=15] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.697922] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:39.697936] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:39.697942] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:39.697953] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:39.697961] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"})
[2023-07-23 11:04:39.697975] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:39.697990] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:39.698009] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.698017] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.698026] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=8][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.698039] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.698052] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=12][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:39.698065] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:39.698079] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:39.698086] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:39.698098] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] query failed(ret=-4122, conn=0x7fd2f9d24050, start=1690081479697794, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:39.698111] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:39.698118] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:39.698158] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=12][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847555})
[2023-07-23 11:04:39.698178] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=19][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847555}, add_timestamp:1690081479695911})
[2023-07-23 11:04:39.698990] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.699005] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=16][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.699014] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999210, timeout_timestamp=1690081480698190)
[2023-07-23 11:04:39.699029] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.699046] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=15][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:39.699057] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:39.699063] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=5][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:39.699072] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=8][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:39.699079] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"})
[2023-07-23 11:04:39.699092] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:39.699106] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:39.699124] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.699133] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.699139] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.699153] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.699168] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:39.699180] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:39.699191] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:39.699197] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:39.699209] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] query failed(ret=-4122, conn=0x7fd5497e8050, start=1690081479698884, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:39.699220] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:39.699227] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:39.699266] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847591})
[2023-07-23 11:04:39.699278] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=12][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847591}, add_timestamp:1690081479695913})
[2023-07-23 11:04:39.699983] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=21][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.699998] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=15][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.700012] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=12][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999323, timeout_timestamp=1690081480699298)
[2023-07-23 11:04:39.700028] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.700042] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:39.700052] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:39.700060] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=7][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:39.700069] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:39.700076] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"})
[2023-07-23 11:04:39.700089] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:39.700103] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=14][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:39.700121] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.700127] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:39.700140] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=13][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.700157] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:39.700170] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:39.700181] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:39.700192] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:39.700199] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:39.700210] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=9][errcode=-4122] query failed(ret=-4122, conn=0x7fd41f004050, start=1690081479699919, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:39.700222] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=12][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:39.700228] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:39.700265] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=10][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847592})
[2023-07-23 11:04:39.700277] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=11][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847592}, add_timestamp:1690081479695919})
[2023-07-23 11:04:39.700300] WDIAG [SERVER] batch_process_tasks (ob_uniq_task_queue.h:498) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=21][errcode=-4122] fail to batch process task(ret=-4122)
[2023-07-23 11:04:39.700306] WDIAG [SERVER] run1 (ob_uniq_task_queue.h:449) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079321-0-0] [lt=6][errcode=-4122] fail to batch execute task(ret=-4122, tasks.count()=4)
[2023-07-23 11:04:39.700602] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.700614] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081572900748, now=[mts=1690081479644693], now0=[mts=1690081479644693], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.700626] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479538282, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.700661] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=30][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:1003}, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60})
[2023-07-23 11:04:39.700676] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=15][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=3222118872, *tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479538282, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.700700] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=23] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039b4a628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479538282, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.700740] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=40][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:501113, schema_version:1689916332789704, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.700748] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.700756] WDIAG [SQL] open (ob_result_set.cpp:157) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.700762] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=4][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:41, local_retry_times:0, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:2, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.700772] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] result set open failed, check if need retry(ret=-4283, cli_ret=-4283, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:39.700783] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=6][errcode=-4283] close result set fail(cret=-4283)
[2023-07-23 11:04:39.700791] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=6][errcode=-4283] execute query fail(ret=-4283, timeout_timestamp=1690081572970748)
[2023-07-23 11:04:39.700798] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=5][errcode=-4283] try to execute again(ret=-4283, type=1, retry_type=2, timeout_remain=93269951)
[2023-07-23 11:04:39.700815] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8] [packet retry query](TRACE=begin_ts=1690081479644646 2023-07-23 03:04:39.644646|[process_begin] u=0 in_queue_time:6673898, receive_ts:1690081472970748, enqueue_ts:1690081479644638|[start_sql] u=1 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=31 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=3 |[check_priv] u=2 |[plan_id] u=1 plan_id:45401|[do_open_plan_begin] u=1 plan_id:45401|[sql_start_stmt_begin] u=1 |[post_packet] u=7 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=56049 |[do_open_plan_end] u=8 |[close_plan_begin] u=23 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=30 |[process_end] u=1 run_ts:1690081479644649|total_timeu=56168)
[2023-07-23 11:04:39.703600] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.703614] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=14][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081479647642], now0=[mts=1690081479647642], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.703625] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:128, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.703657] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=30][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:39.703673] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=16][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:128, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.703702] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=27] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479642532, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:128, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.703746] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=44][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.703756] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.703762] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.703770] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:39.703775] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:39.703784] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=125, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:39.703796] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10] will sleep(sleep_us=100000, remain_us=15466052, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=125, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:39.712669] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3361][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.713019] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3358][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.713304] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3360][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.713572] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3364][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.713575] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3359][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.713838] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3365][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.713988] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3363][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.714012] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3362][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.714062] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3357][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.716733] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=0][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.716744] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.716749] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:39.716754] WDIAG [SQL.ENG] refresh_statistics (ob_px_tenant_target_monitor.cpp:114) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4122] query statistics failed(ret=-4122)
[2023-07-23 11:04:39.716758] WDIAG [COMMON] operator() (ob_px_target_mgr.h:143) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4122] target monitor refresh statistics failed(ret=-4122, px_tenant_info={v:1}, px_res_info->get_target_monitor()={is_init:true, tenant_id:1, server:"192.168.1.1:1001", dummy_cache_leader:"10.1.129.154:1001", role:2})
[2023-07-23 11:04:39.716772] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.716777] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.716781] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:39.716787] WDIAG [SQL.ENG] refresh_statistics (ob_px_tenant_target_monitor.cpp:114) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4122] query statistics failed(ret=-4122)
[2023-07-23 11:04:39.716790] WDIAG [COMMON] operator() (ob_px_target_mgr.h:143) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4122] target monitor refresh statistics failed(ret=-4122, px_tenant_info={v:1001}, px_res_info->get_target_monitor()={is_init:true, tenant_id:1001, server:"192.168.1.1:1001", dummy_cache_leader:"10.1.129.154:1001", role:2})
[2023-07-23 11:04:39.716932] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.716938] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:39.716943] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:39.716947] WDIAG [SQL.ENG] refresh_statistics (ob_px_tenant_target_monitor.cpp:114) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4122] query statistics failed(ret=-4122)
[2023-07-23 11:04:39.716950] WDIAG [COMMON] operator() (ob_px_target_mgr.h:143) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4122] target monitor refresh statistics failed(ret=-4122, px_tenant_info={v:1002}, px_res_info->get_target_monitor()={is_init:true, tenant_id:1002, server:"192.168.1.1:1001", dummy_cache_leader:"10.1.129.154:1001", role:2})
[2023-07-23 11:04:39.720083] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3372][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.720245] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3367][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.720482] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3369][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=15] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.720602] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3366][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.720604] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3349][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=86/115, request doing=0/0)
[2023-07-23 11:04:39.720734] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3341][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=68/69, request doing=0/0)
[2023-07-23 11:04:39.720804] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3339][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=72/111, request doing=0/0)
[2023-07-23 11:04:39.720889] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3371][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.720893] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3374][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.720926] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3340][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=63/115, request doing=0/0)
[2023-07-23 11:04:39.720940] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3368][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.720942] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3347][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=61/93, request doing=0/0)
[2023-07-23 11:04:39.720982] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3370][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.721228] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3345][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=87/100, request doing=0/0)
[2023-07-23 11:04:39.721238] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3348][BatchIO][T0][Y0-0000000000000000-0-0] [lt=4] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=67/82, request doing=0/0)
[2023-07-23 11:04:39.721280] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3346][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=75/102, request doing=0/0)
[2023-07-23 11:04:39.721383] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3352][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=80/105, request doing=0/0)
[2023-07-23 11:04:39.721400] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3344][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=87/104, request doing=0/0)
[2023-07-23 11:04:39.721439] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3376][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.721560] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3342][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=79/92, request doing=0/0)
[2023-07-23 11:04:39.721589] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3343][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=80/96, request doing=0/0)
[2023-07-23 11:04:39.721648] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3355][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:39.721646] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3353][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=89/110, request doing=0/0)
[2023-07-23 11:04:39.721751] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3350][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=68/114, request doing=0/0)
[2023-07-23 11:04:39.721839] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3351][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=73/90, request doing=0/0)
[2023-07-23 11:04:39.721984] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3354][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=66/85, request doing=0/0)
[2023-07-23 11:04:39.724963] INFO  [CLOG] operator() (ob_log_replay_service.cpp:1471) [83357][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=11] get_replay_process success(id={id:1}, replayed_log_size=99688816093, unreplayed_log_size=0)
[2023-07-23 11:04:39.724994] INFO  [CLOG] runTimerTask (ob_log_replay_service.cpp:154) [83357][T1001_ReplayPro][T1001][Y0-0000000000000000-0-0] [lt=10] dump tenant replay process(tenant_id=1001, unreplayed_log_size(MB)=0, estimate_time(second)=0, replayed_log_size(MB)=95070, last_replayed_log_size(MB)=95070, round_cost_time(second)=10, pending_replay_log_size(MB)=0)
[2023-07-23 11:04:39.727281] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [3727][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(evict_high_mem_level=412316860, evict_high_size_level=9000000, evict_batch_count=0, elapse_time=0, size_used=88141, mem_used=232931328)
[2023-07-23 11:04:39.730252] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.737198] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B08-0-0] [lt=1][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:39.737216] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B08-0-0] [lt=19][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.737232] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B08-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.739182] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:89, local_retry_times:89, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.739214] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=24][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.739224] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.739233] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.739254] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.739334] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.743655] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479743642]})
[2023-07-23 11:04:39.743677] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.743680] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=1] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479743672]})
[2023-07-23 11:04:39.743686] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.743697] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.743703] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.743862] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.743872] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.743882] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=6][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.743900] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479743891]})
[2023-07-23 11:04:39.743904] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=21][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.743925] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=21] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.743945] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=18] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.743954] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B059-0-0] [lt=7] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479743952, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.744112] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F1-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.744126] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.744517] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C22-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.745583] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=44] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479745576})
[2023-07-23 11:04:39.745589] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC31-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.746201] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=13][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.746217] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.746334] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479745510 2023-07-23 03:04:39.745510|[process_begin] u=0 in_queue_time:3792189, receive_ts:1690081475953320, enqueue_ts:1690081479745501|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=4 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=49 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=28 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=7 |[do_open_plan_end] u=0 |[get_das_id_begin] u=31 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=505 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=8 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=41 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=5 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=24 |[close_das_task_begin] u=2 |[close_das_task_end] u=25 |[start_end_stmt] u=26 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=16 |[process_end] u=1 run_ts:1690081479745515|total_timeu=823)
[2023-07-23 11:04:39.752858] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73136676044, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:39.754015] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=14] Wash time detail, (compute_wash_size_time=80, refresh_score_time=673, wash_time=470)
[2023-07-23 11:04:39.755731] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=25][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:39.755743] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=12][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196758}, scn:1690081479743568}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196758}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479644637, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.755773] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=30][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196758}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479644637, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.755928] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=154][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:39.755935] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.755940] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:39.755948] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.755954] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:39.755960] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:39.755964] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:39.755991] INFO  [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:44) [3128][ServerGTimer][T0][YB420A01819C-0005FDE72B4C6664-0-0] [lt=12] dump dtl interm result cost(us)(dump_cost=17877, ret=0, interm count=0, dump count=352531)
[2023-07-23 11:04:39.755969] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503986, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196758}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479644637, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196758}, scn:1690081479743568}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:39.756374] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=405][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:39.756379] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:39.756387] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:39.756392] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:39.756395] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:39.756402] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:39.756409] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:39.756413] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:39.756418] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:39.756422] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:39.756428] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:39.756432] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:39.756450] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:67, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:39.756463] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196758}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479644637, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.756502] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194C-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.756494] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=30] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196758}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479644637, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.756517] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=23] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479743568, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196758}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479644637, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.756547] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:39.756553] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:39.756557] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:39.756564] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=92903571)
[2023-07-23 11:04:39.756580] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081479755625 2023-07-23 03:04:39.755625|[process_begin] u=0 in_queue_time:7095490, receive_ts:1690081472660135, enqueue_ts:1690081479755616|[start_sql] u=1 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=56 |[check_priv] u=7 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=2 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=241 |[sync_auto_value] u=433 |[close_plan_begin] u=58 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=83 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=31 |[process_end] u=1 run_ts:1690081479755628|total_timeu=954)
[2023-07-23 11:04:39.756931] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.756939] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.756956] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14295024, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.756966] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.757445] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F0-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.758579] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F0-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.759570] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F0-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.760443] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:167) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=7] sql audit evict mem start(evict_low_mem_level=28923920, evict_high_mem_level=49895441, size_used=41, mem_used=60312576)
[2023-07-23 11:04:39.760461] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:173) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=10] release old cannot free more memory
[2023-07-23 11:04:39.760468] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=7] sql audit evict task end(evict_high_mem_level=49895441, evict_high_size_level=90000, evict_batch_count=1, elapse_time=26, size_used=0, mem_used=60312576)
[2023-07-23 11:04:39.760550] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F0-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.773449] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=19422, clean_start_pos=8600000, clean_num=200000)
[2023-07-23 11:04:39.773749] INFO  [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:59) [3128][ServerGTimer][T0][YB420A01819C-0005FDE72B4C6664-0-0] [lt=12] clear dtl interm result cost(us)(clear_cost=17746, ret=0, expire_keys_.count()=0, dump count=352531, interm count=0, clean count=0)
[2023-07-23 11:04:39.773782] INFO  [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:241) [3128][ServerGTimer][T0][YB420A01819C-0005FDE72B4C6664-0-0] [lt=11] set tenant work area memory(tenant_id=1001, wa_pctg=5, limit=157286400, ret=0)
[2023-07-23 11:04:39.773822] INFO  [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:241) [3128][ServerGTimer][T0][YB420A01819C-0005FDE72B4C6664-0-0] [lt=7] set tenant work area memory(tenant_id=1002, wa_pctg=5, limit=1063004405, ret=0)
[2023-07-23 11:04:39.774203] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3128][ServerGTimer][T1][YB420A01819C-0005FDE72B4C6664-0-0] [lt=5][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.774215] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3128][ServerGTimer][T1][YB420A01819C-0005FDE72B4C6664-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.774281] INFO  [PALF] get_disk_usage (palf_env_impl.cpp:781) [3128][ServerGTimer][T1002][YB420A01819C-0005FDE72B4C6664-0-0] [lt=0] get_disk_usage(ret=0, capacity(MB):=60825, used(MB):=48597)
[2023-07-23 11:04:39.776322] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.776434] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479775784 2023-07-23 03:04:39.775784|[process_begin] u=0 in_queue_time:3751958, receive_ts:1690081476023826, enqueue_ts:1690081479775775|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=29 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=430 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=8 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[remote_task_completed] u=29 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=1 |[close_plan_begin] u=55 |[close_das_task_begin] u=1 |[close_das_task_end] u=23 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[query_end] u=8 |[process_end] u=1 run_ts:1690081479775787|total_timeu=649)
[2023-07-23 11:04:39.777348] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [83138][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=9] sql audit evict task end(evict_high_mem_level=510242114, evict_high_size_level=9000000, evict_batch_count=0, elapse_time=0, size_used=124291, mem_used=415948800)
[2023-07-23 11:04:39.792443] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1}, delta=14276719, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.792463] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=11] get wrs ts(ls_id={id:1}, delta=14385404, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.792472] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.792468] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=22] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.792498] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=21] get wrs ts(ls_id={id:1001}, delta=14385166, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.792515] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.792541] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1002}, delta=14714453, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.792557] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.792582] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1003}, delta=14648594, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.792599] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.794757] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [3732][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=5] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000)
[2023-07-23 11:04:39.794770] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [3732][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0)
[2023-07-23 11:04:39.794777] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:214) [3732][T1_ObTimer][T1][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=9384)
[2023-07-23 11:04:39.795083] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.795094] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081479739325], now0=[mts=1690081479739325], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.795104] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908677}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479738316, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.795131] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=24][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:39.795143] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908677}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479738316, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.795165] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=20] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479642535, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908677}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479738316, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.795199] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=34][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.795208] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.795215] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.795221] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:39.795225] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:39.795233] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=90, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:39.795242] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8] will sleep(sleep_us=90000, remain_us=20881295, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=90, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:39.796028] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=49][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479743952})
[2023-07-23 11:04:39.796068] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=39][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479743952}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.796105] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=36][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.796125] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=19][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.796138] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479743952}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:39.796188] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=49][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:39.796211] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=22][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:39.796226] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:39.796244] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:39.799317] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:39.799408] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=39][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479743952})
[2023-07-23 11:04:39.799454] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=45][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479743952}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.799492] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=37][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.799509] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=16][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.799522] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.799537] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:39.799549] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:39.799563] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.799577] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=13][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:39.799589] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:39.799604] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:39.799615] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:39.799627] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:39.799639] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:39.799655] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:39.799668] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:39.799681] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:39.799693] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:39.799710] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:46, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:39.799730] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.799748] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=5424943)
[2023-07-23 11:04:39.799795] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=15][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541948671493, cache_obj->added_lc()=false, cache_obj->get_object_id()=20499, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:39.799832] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18] [packet retry query](TRACE=begin_ts=1690081479795964 2023-07-23 03:04:39.795964|[process_begin] u=0 in_queue_time:4571273, receive_ts:1690081475224690, enqueue_ts:1690081479795952|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=50 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[pc_choose_plan] u=216 |[transform_with_outline_begin] u=129 |[transform_with_outline_end] u=1 |[resolve_begin] u=5 |[resolve_end] u=1021 |[transform_begin] u=39 |[optimize_begin] u=1604 |[tl_calc_part_id_end] u=244 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=425 |[query_end] u=120 |[process_end] u=1 run_ts:1690081479795969|total_timeu=3866)
[2023-07-23 11:04:39.803863] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:125, local_retry_times:125, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.803878] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.803881] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.803884] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.803895] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.803936] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.803944] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.805254] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=16584, replace_start_pos=22300000, replace_num=100000)
[2023-07-23 11:04:39.806172] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:39.806183] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196759}, scn:1690081479743571}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196759}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479695181, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.806215] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=31][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196759}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479695181, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.806370] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=154][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:39.806376] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.806381] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:39.806387] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.806393] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:39.806397] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:39.806403] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:39.806409] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503987, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196759}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479695181, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196759}, scn:1690081479743571}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:39.806801] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=392][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:39.806806] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:39.806814] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:39.806818] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:39.806821] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:39.806826] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:39.806833] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:39.806837] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:39.806842] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:39.806846] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:39.806850] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:39.806856] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:39.806879] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:62, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:39.806893] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196759}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479695181, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.806920] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196759}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479695181, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.806945] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=23] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479743571, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196759}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479695181, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.806974] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:39.806981] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:39.806985] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:39.806991] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=93468249)
[2023-07-23 11:04:39.807005] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081479806078 2023-07-23 03:04:39.806078|[process_begin] u=0 in_queue_time:6530838, receive_ts:1690081473275239, enqueue_ts:1690081479806066|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=45 |[check_priv] u=8 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=1 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=6 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=2 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=240 |[sync_auto_value] u=417 |[close_plan_begin] u=63 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=80 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=31 |[process_end] u=1 run_ts:1690081479806080|total_timeu=927)
[2023-07-23 11:04:39.807385] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079322-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.807906] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079322-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.808398] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079322-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.808878] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079322-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.816224] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=1][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.830308] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.836311] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [83154][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}])
[2023-07-23 11:04:39.844399] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479844388]})
[2023-07-23 11:04:39.844418] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479844411]})
[2023-07-23 11:04:39.844416] INFO  [PALF] check_parent_health (log_config_mgr.cpp:2351) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=68] re_register_parent reason(palf_id=1002, self="192.168.1.1:1001", is_registering_timeout=false, first_registration=true, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=-1, last_first_register_time_us=1690081479844353, register_time_us=-1, curr_time_us=1690081479844412)
[2023-07-23 11:04:39.844440] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=16][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.844443] WDIAG [PALF] check_parent_health (log_config_mgr.cpp:2354) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=26][errcode=-4023] register request timeout, re_register_parent failed(ret=-4023, ret="OB_EAGAIN", palf_id=1002, self="192.168.1.1:1001")
[2023-07-23 11:04:39.844461] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.844475] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=12][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.844498] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.844509] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.844573] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479844565]})
[2023-07-23 11:04:39.844680] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=15][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.844743] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=22][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.844789] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F2-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.844766] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=15][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.844805] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=38][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.844840] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=34] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.844879] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=34] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.844896] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05A-0-0] [lt=12] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479844892, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.845050] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C30-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.846481] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=44] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479846474})
[2023-07-23 11:04:39.846486] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC32-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.847072] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.847182] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479846408 2023-07-23 03:04:39.846408|[process_begin] u=0 in_queue_time:3893088, receive_ts:1690081475953320, enqueue_ts:1690081479846399|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=50 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=28 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[exec_plan_end] u=6 |[do_open_plan_end] u=0 |[get_das_id_begin] u=37 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=15 |[calc_partition_location_begin] u=468 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:427|[do_open_plan_begin] u=7 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=32 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=24 |[close_das_task_begin] u=1 |[close_das_task_end] u=28 |[start_end_stmt] u=16 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=15 |[process_end] u=1 run_ts:1690081479846411|total_timeu=773)
[2023-07-23 11:04:39.850410] INFO  [STORAGE] operator() (ob_tenant_freezer.cpp:125) [83291][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=7] ====== tenant freeze timer task ======
[2023-07-23 11:04:39.850872] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83291][T1001_Occam][T1001][YB420A01819C-0005FDE720591A00-0-0] [lt=11][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.850886] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83291][T1001_Occam][T1001][YB420A01819C-0005FDE720591A00-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.850935] WDIAG [STORAGE] check_and_freeze_normal_data_ (ob_tenant_freezer.cpp:408) [83291][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=0] [TenantFreezer] fail to do major freeze(tmp_ret=-4122)
[2023-07-23 11:04:39.856999] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.857047] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14395752, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.857058] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194D-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.857067] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.857381] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F3-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.859780] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.859792] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081479803932], now0=[mts=1690081479803932], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.859801] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:129, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.859825] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=23][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:39.859841] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=15][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:129, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.859862] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=20] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479743570, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:129, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.859898] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=35][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.859911] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.859916] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.859919] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:39.859924] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:39.859932] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=126, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:39.859940] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8] will sleep(sleep_us=100000, remain_us=15309908, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=126, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:39.865409] INFO  [PALF] check_parent_health (log_config_mgr.cpp:2351) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=13] re_register_parent reason(palf_id=1001, self="192.168.1.1:1001", is_registering_timeout=false, first_registration=true, parent_timeout=false, parent_keepalive_time_us=-1, last_submit_register_req_time_us=-1, last_first_register_time_us=1690081479865372, register_time_us=-1, curr_time_us=1690081479865407)
[2023-07-23 11:04:39.865425] WDIAG [PALF] check_parent_health (log_config_mgr.cpp:2354) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15][errcode=-4023] register request timeout, re_register_parent failed(ret=-4023, ret="OB_EAGAIN", palf_id=1001, self="192.168.1.1:1001")
[2023-07-23 11:04:39.866679] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=32][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:39.866689] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039b45418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196760}, scn:1690081479844355}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196760}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479756211, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.866713] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=25][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196760}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479756211, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.866828] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=114][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:39.866834] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.866837] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:39.866842] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.866845] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=2][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:39.866847] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:39.866851] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:39.866854] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503988, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196760}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479756211, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039b4a628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196760}, scn:1690081479844355}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:39.867146] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=292][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:39.867150] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:39.867157] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:39.867159] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:39.867162] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:39.867165] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:39.867169] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:39.867172] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:39.867175] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:39.867178] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=2][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:39.867180] WDIAG [SQL] open (ob_result_set.cpp:159) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=2][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:39.867184] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:39.867200] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:68, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:39.867212] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196760}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479756211, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.867234] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=22] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196760}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479756211, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.867253] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=18] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479844355, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196760}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479756211, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.867275] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=19][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:39.867280] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:39.867283] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:39.867288] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=92792847)
[2023-07-23 11:04:39.867304] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9] [packet retry query](TRACE=begin_ts=1690081479866590 2023-07-23 03:04:39.866590|[process_begin] u=0 in_queue_time:7206455, receive_ts:1690081472660135, enqueue_ts:1690081479866580|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=41 |[check_priv] u=6 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[insert_start] u=14 |[get_das_id_begin] u=10 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=178 |[sync_auto_value] u=311 |[close_plan_begin] u=42 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=63 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=28 |[process_end] u=0 run_ts:1690081479866592|total_timeu=713)
[2023-07-23 11:04:39.867667] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F4-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.868131] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F4-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.868589] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F4-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.869035] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F4-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.872082] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=10][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.872096] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=14][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081572900748, now=[mts=1690081479816216], now0=[mts=1690081479816216], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.872110] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=14][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479700450, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.872148] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=33][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:1003}, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60})
[2023-07-23 11:04:39.872166] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=18][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=3222118872, *tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479700450, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:39.872199] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=31] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479700450, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.872249] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=49][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:501113, schema_version:1689916332789704, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.872262] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.872269] WDIAG [SQL] open (ob_result_set.cpp:157) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=5][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.872278] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:42, local_retry_times:0, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:2, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.872296] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=12][errcode=-4283] result set open failed, check if need retry(ret=-4283, cli_ret=-4283, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:39.872313] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4283] close result set fail(cret=-4283)
[2023-07-23 11:04:39.872322] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] execute query fail(ret=-4283, timeout_timestamp=1690081572970748)
[2023-07-23 11:04:39.872334] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] try to execute again(ret=-4283, type=1, retry_type=2, timeout_remain=93098415)
[2023-07-23 11:04:39.872355] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9] [packet retry query](TRACE=begin_ts=1690081479816169 2023-07-23 03:04:39.816169|[process_begin] u=0 in_queue_time:6845421, receive_ts:1690081472970748, enqueue_ts:1690081479816160|[start_sql] u=1 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=30 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=3 |[check_priv] u=4 |[plan_id] u=0 plan_id:45401|[do_open_plan_begin] u=2 plan_id:45401|[sql_start_stmt_begin] u=0 |[post_packet] u=8 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=56040 |[do_open_plan_end] u=7 |[close_plan_begin] u=39 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=40 |[process_end] u=1 run_ts:1690081479816172|total_timeu=56184)
[2023-07-23 11:04:39.872931] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83283][T1001_LogUpdate][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.873393] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F4-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.876788] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4015] sock error detect by epoll(mask=8221, *s={this:0x7fd42142c690, fd:536, err:5, last_decode_time_:1690077703840703, last_write_time_:1690077703841273, read_buffer_.get_consume_sz():391, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.876811] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3222102956, proxy_sessid=720999913969917982, server_id=3, ret=0)
[2023-07-23 11:04:39.876844] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:795) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=9] sock ref clean, do destroy(*s={this:0x7fd42142c690, fd:536, err:5, last_decode_time_:1690077703840703, last_write_time_:1690077703841273, read_buffer_.get_consume_sz():391, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:39.876862] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=8] unlock session of tenant(conn.sessid_=3222102956, proxy_sessid=720999913969917982, conn.tenant_id_=1002)
[2023-07-23 11:04:39.876870] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3387][sql_nio9][T0][Y0-0000000000000000-0-0] [lt=8] connection close(sessid=3222102956, proxy_sessid=720999913969917982, tenant_id=1002, server_id=3, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:39.876889] INFO  [SERVER] run (obmp_disconnect.cpp:78) [83222][T1002_L0_G0][T1002][Y0-0000000000000000-0-0] [lt=9] free session successfully(sessid=3222102956)
[2023-07-23 11:04:39.877205] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.877328] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479876657 2023-07-23 03:04:39.876657|[process_begin] u=0 in_queue_time:3852830, receive_ts:1690081476023826, enqueue_ts:1690081479876647|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=4 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=34 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=4 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=438 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=34 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=54 |[close_das_task_begin] u=1 |[close_das_task_end] u=30 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=9 |[process_end] u=1 run_ts:1690081479876662|total_timeu=670)
[2023-07-23 11:04:39.885306] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:90, local_retry_times:90, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.885325] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.885334] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.885338] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.885347] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.885402] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.892495] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.892513] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.892523] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.892519] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.892546] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14485190, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.892557] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.892569] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.892566] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14376505, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.892583] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=13][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.892585] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.892602] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.892608] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.892613] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get wrs ts(ls_id={id:1001}, delta=14484952, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.892626] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.892652] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1002}, delta=14814239, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.892669] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.892731] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=38] get wrs ts(ls_id={id:1003}, delta=14748380, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.892750] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=16] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.892895] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=12][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.892911] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=15][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.892929] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=12][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.892961] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=31][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.892989] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=27] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.893022] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=30] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.893035] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05B-0-0] [lt=9] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479893032, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.893232] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.896916] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.896958] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479896836 2023-07-23 03:04:39.896836|[process_begin] u=0 in_queue_time:4672146, receive_ts:1690081475224690, enqueue_ts:1690081479896827|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=26 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=2 |[parse_job_end] u=10 |[remote_task_completed] u=42 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29251}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=11 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=2 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=9 |[process_end] u=1 run_ts:1690081479896839|total_timeu=121)
[2023-07-23 11:04:39.912780] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [3743][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=23] start do ls ha handler(ls_id_array_=[{id:1}])
[2023-07-23 11:04:39.917212] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=36][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:39.917231] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=18][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196761}, scn:1690081479892760}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196761}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479806596, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.917307] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=75][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196761}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479806596, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.917576] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=267][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:39.917590] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=13][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.917602] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:39.917615] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=13][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.917625] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:39.917634] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:39.917644] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:39.917746] INFO  [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:374) [3699][T1_TenantMetaMe][T1][Y0-0000000000000000-0-0] [lt=30] Recycle 0 table(ret=0, allocator_={used:1589877, total:4666096}, tablet_pool_={typeid(T).name():"N9oceanbase7storage8ObTabletE", sizeof(T):2560, used_obj_cnt:797, free_obj_hold_cnt:12, allocator used:2122816, allocator total:2223872}, sstable_pool_={typeid(T).name():"N9oceanbase12blocksstable9ObSSTableE", sizeof(T):1088, used_obj_cnt:1570, free_obj_hold_cnt:1658, allocator used:3718656, allocator total:3859072}, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):3008, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1920, used_obj_cnt:36, free_obj_hold_cnt:766, allocator used:1591168, allocator total:1700608}, tablet count=797, min_minor_cnt=4, pinned_tablet_cnt=0)
[2023-07-23 11:04:39.917654] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503989, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196761}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479806596, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196761}, scn:1690081479892760}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:39.918366] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=712][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:39.918375] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:39.918389] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=12][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:39.918399] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:39.918408] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:39.918418] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:39.918430] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:39.918439] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:39.918449] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:39.918455] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:39.918466] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:39.918476] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:39.918532] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:63, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:39.918557] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=15] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196761}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479806596, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.918608] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=50] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196761}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479806596, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.918655] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=45] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479892760, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196761}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479806596, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.918706] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=45][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:39.918718] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:39.918728] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=10][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:39.918741] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=10][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=93356499)
[2023-07-23 11:04:39.918765] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=10] [packet retry query](TRACE=begin_ts=1690081479917041 2023-07-23 03:04:39.917041|[process_begin] u=0 in_queue_time:6641800, receive_ts:1690081473275239, enqueue_ts:1690081479917028|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=82 |[check_priv] u=14 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=3 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=5 |[exec_plan_begin] u=0 |[exec_plan_end] u=4 |[do_open_plan_end] u=0 |[insert_open] u=3 |[insert_start] u=17 |[get_das_id_begin] u=20 |[get_das_id_end] u=0 |[insert_end] u=5 |[do_local_das_task_begin] u=3 |[do_local_das_task_end] u=448 |[sync_auto_value] u=764 |[close_plan_begin] u=130 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=148 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=4 |[query_end] u=58 |[process_end] u=2 run_ts:1690081479917045|total_timeu=1723)
[2023-07-23 11:04:39.918981] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079323-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.919511] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.920527] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F1-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.921500] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F1-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.930361] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:39.941217] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:39.941230] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081479885395], now0=[mts=1690081479885395], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:39.941242] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908678}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479884351, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.941273] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=28][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:39.941287] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=15][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908678}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479884351, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.941319] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=30] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479844357, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908678}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479884351, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:39.941359] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=40][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:39.941370] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:39.941377] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:39.941383] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:39.941389] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:39.941399] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=91, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:39.941409] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9] will sleep(sleep_us=91000, remain_us=20735129, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=91, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:39.944919] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081479944907]})
[2023-07-23 11:04:39.944932] INFO  [STORAGE.TRANS] operator() (ob_ts_mgr.h:171) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts functor(ret=-4122, ret="OB_RPC_POST_ERROR", gts_tenant_info={v:1})
[2023-07-23 11:04:39.944945] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081479944939]})
[2023-07-23 11:04:39.944944] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.944953] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.944967] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.944973] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.945085] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081479945077]})
[2023-07-23 11:04:39.945189] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.945200] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.945213] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=8][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.945235] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=21][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.945257] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=21] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.945278] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=19] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.945288] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05C-0-0] [lt=7] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081479945286, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:39.945393] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.945558] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C31-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.945573] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C31-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.947321] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=32] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081479947315})
[2023-07-23 11:04:39.947326] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC33-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:39.947892] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.947995] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479947246 2023-07-23 03:04:39.947246|[process_begin] u=0 in_queue_time:3993925, receive_ts:1690081475953320, enqueue_ts:1690081479947234|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=54 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[plc_serialize_begin] u=0 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=26 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=7 |[do_open_plan_end] u=0 |[get_das_id_begin] u=49 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=14 |[calc_partition_location_begin] u=440 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=5 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=6 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=28 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=22 |[close_das_task_begin] u=1 |[close_das_task_end] u=22 |[start_end_stmt] u=20 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=14 |[process_end] u=1 run_ts:1690081479947249|total_timeu=748)
[2023-07-23 11:04:39.957046] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.957069] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14495683, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.957080] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.957667] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.957681] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.957689] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.957702] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:39.957708] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.957810] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.957854] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=6][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:39.957865] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:39.957877] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=8][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.957899] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=22][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.957916] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=17] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:39.957940] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=20] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:39.958419] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BD-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.960004] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:126, local_retry_times:126, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:39.960021] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=14][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.960026] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:39.960031] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.960042] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:39.960090] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.973691] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73136676044, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:39.974306] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3633][T1_LogUpdater][T1][Y0-0000000000000000-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.974849] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash time detail, (compute_wash_size_time=83, refresh_score_time=673, wash_time=466)
[2023-07-23 11:04:39.977674] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=49][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:39.977688] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=13][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196762}, scn:1690081479956873}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196762}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479866428, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.977724] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=36][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196762}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479866428, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:39.977912] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=187][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:39.977921] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.977927] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:39.977939] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=12][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:39.977944] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:39.977951] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:39.977959] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:39.978039] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.978114] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081479977510 2023-07-23 03:04:39.977510|[process_begin] u=0 in_queue_time:3953683, receive_ts:1690081476023826, enqueue_ts:1690081479977501|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=29 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=426 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=36 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=20 |[close_das_task_begin] u=1 |[close_das_task_end] u=22 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=8 |[process_end] u=0 run_ts:1690081479977512|total_timeu=603)
[2023-07-23 11:04:39.977967] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503990, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196762}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479866428, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196762}, scn:1690081479956873}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:39.978474] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=507][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:39.978480] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:39.978490] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:39.978495] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:39.978502] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:39.978510] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:39.978519] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:39.978526] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:39.978534] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:39.978539] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:39.978546] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:39.978552] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:39.978576] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:69, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:39.978594] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=11] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196762}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479866428, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.978630] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=36] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196762}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479866428, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:39.978663] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=32] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479956873, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196762}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479866428, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:39.978699] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=31][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:39.978709] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:39.978716] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:39.978725] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=92681410)
[2023-07-23 11:04:39.978743] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081479977545 2023-07-23 03:04:39.977545|[process_begin] u=0 in_queue_time:7317409, receive_ts:1690081472660135, enqueue_ts:1690081479977530|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=64 |[check_priv] u=10 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=3 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=4 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[insert_open] u=2 |[value_start_calc_row] u=3 |[value_after_calc_row] u=10 |[insert_start] u=0 |[get_das_id_begin] u=16 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=297 |[sync_auto_value] u=543 |[close_plan_begin] u=78 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=105 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=43 |[process_end] u=1 run_ts:1690081479977548|total_timeu=1197)
[2023-07-23 11:04:39.979237] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.979916] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.980581] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.981216] INFO  [PALF] is_pending_log_clear_ (log_state_mgr.cpp:747) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] follower pending log not clear, need wait(bool_ret=false, palf_id=1003, last_slide_end_lsn={lsn:375365428166}, pending_end_lsn={lsn:375365428400})
[2023-07-23 11:04:39.981223] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.987693] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=33][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.987706] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:39.989795] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.989972] INFO  [CLOG] handle_sync_request (log_request_handler.cpp:486) [83648][T1002_L0_G1][T1002][YB420A01819C-0005FDE7216A81B5-0-0] [lt=9] i am not leader(ret=-4038, palf_id=1003, req={get_type:1}, role=2, is_pending_state=true)
[2023-07-23 11:04:39.990040] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4038] execute rpc fail(ret=-4038, dst="192.168.1.1:1001")
[2023-07-23 11:04:39.990063] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=23][errcode=-4038] 
[2023-07-23 11:04:39.990085] WDIAG [CLOG] post_sync_request_to_server_ (log_net_service.h:291) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4038] ObLogRpc post_sync_request failed(ret=-4038, palf_id=1003, req={get_type:1}, server="192.168.1.1:1001")
[2023-07-23 11:04:39.990102] WDIAG [CLOG] get_leader_max_scn_ (palf_handle_impl.cpp:4223) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4038] get_palf_max_scn failed(ret=-4038, palf_id=1003)
[2023-07-23 11:04:39.990118] WDIAG [CLOG] is_in_sync_ (palf_handle_impl.cpp:4182) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4038] get_palf_max_scn failed(ret=-4038, self="192.168.1.1:1001", palf_id=1003)
[2023-07-23 11:04:39.990307] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3E-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.990322] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.990329] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF2-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.990345] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF2-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.990432] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C23-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:39.990448] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C23-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.990582] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C32-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:39.992559] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1}, delta=14476427, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.992566] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:39.992573] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.992581] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14585112, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.992590] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.992594] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get wrs ts(ls_id={id:1001}, delta=14584874, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.992601] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.992619] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] get wrs ts(ls_id={id:1002}, delta=14914161, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.992627] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.992640] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1003}, delta=14848302, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:39.992648] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:39.994105] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=19244, clean_start_pos=8800000, clean_num=200000)
[2023-07-23 11:04:39.997734] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=42][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479945286})
[2023-07-23 11:04:39.997754] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=20][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479945286}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.997769] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=16][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.997775] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.997780] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479945286}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:39.997807] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=27][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:39.997818] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:39.997822] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:39.997828] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:39.999357] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:39.999401] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479945286})
[2023-07-23 11:04:39.999420] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=19][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081479945286}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:39.999438] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=17][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.999449] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:39.999453] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:39.999461] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:39.999465] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:39.999472] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.999479] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:39.999483] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:39.999489] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:39.999493] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:39.999497] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:39.999501] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:39.999508] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:39.999516] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:39.999522] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:39.999529] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:39.999536] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:48, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:39.999545] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:39.999553] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=5225138)
[2023-07-23 11:04:39.999578] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541948871282, cache_obj->added_lc()=false, cache_obj->get_object_id()=20500, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:39.999597] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9] [packet retry query](TRACE=begin_ts=1690081479997701 2023-07-23 03:04:39.997701|[process_begin] u=0 in_queue_time:4773011, receive_ts:1690081475224690, enqueue_ts:1690081479997692|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=25 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=94 |[transform_with_outline_begin] u=54 |[transform_with_outline_end] u=1 |[resolve_begin] u=3 |[resolve_end] u=526 |[transform_begin] u=20 |[optimize_begin] u=768 |[tl_calc_part_id_end] u=148 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=188 |[query_end] u=60 |[process_end] u=1 run_ts:1690081479997704|total_timeu=1895)
[2023-07-23 11:04:40.002224] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2135) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF_DUMP](palf_id=1, self="192.168.1.1:1001", [SlidingWindow]={palf_id:1, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:32949107, max_lsn:8392989598, max_scn:{val:1690081465515197326}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:8392989598}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:8392989476}, last_submit_end_lsn:{lsn:8392989598}, last_submit_log_id:32949107, last_submit_log_pid:1, max_flushed_lsn:{lsn:8392989476}, max_flushed_end_lsn:{lsn:8392989598}, max_flushed_log_pid:1, committed_end_lsn:{lsn:8392989598}, last_slide_log_id:32949107, last_slide_scn:{val:1690081465515197326}, last_slide_lsn:{lsn:8392989476}, last_slide_end_lsn:{lsn:8392989598}, last_slide_log_pid:1, last_slide_log_accum_checksum:383597832, last_fetch_end_lsn:{lsn:8434932638}, last_fetch_max_log_id:32951155, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd501ff2530}, [StateMgr]={this:0x7fd501ff53f0, self:"192.168.1.1:1001", palf_id:1, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"0.0.0.0:0", leader_epoch:0, is_sync_enabled:true, allow_vote:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:32949108, is_changing_config_with_arb:false, reconfirm_start_time_us:-1, palf_role_change_cb:0x7fd501ffb670, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"192.168.1.1:1001", alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, prev_log_proposal_id:9223372036854775807, applied_alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, applied_alive_paxos_replica_num:3, applied_all_learnerlist:{learner_num:0, learner_array:[]}, prev_lsn:{lsn:18446744073709551615}, prev_mode_pid:9223372036854775807, state:0, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, region:"default_region", paxos_member_region_map:{}, register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7fd501ff35b0}, [ModeMgr]={palf_id:1, self:"192.168.1.1:1001", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1689934475974181485}, min_block_id:119, base_lsn_for_block_gc:{lsn:8388096000}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:1, base_lsn:{lsn:8388096000}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}}, log_meta_storage:{log_tail:{lsn:499712}, readable_log_tail:{lsn:499712}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:8455200768}, min_scn:{val:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1/meta", dir_fd:475, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:66605056, block_header_serialize_buf_:0x7fd501ff7030}, log_storage:{log_tail:{lsn:8392989598}, readable_log_tail:{lsn:8392989598}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:8388096000}, min_scn:{val:1690079128200244937}, curr_block_id:125, palf_id:1}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1/log", dir_fd:476, min_block_id:119, max_block_id:126, curr_writable_block_id:125}, logical_block_size_:67104768, curr_block_writable_size_:62211170, block_header_serialize_buf_:0x7fd501ff8cb0}, palf_epoch:1, this:0x7fd501ff5af0}, [Reconfirm]={palf_id:1, self:"192.168.1.1:1001", state:"INITED", new_proposal_id:9223372036854775807, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:0, majority_max_log_server:"0.0.0.0:0", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:18446744073709551615}, saved_end_lsn:{lsn:18446744073709551615}, last_submit_prepare_req_time_us:-1, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, has_notify_fetch:false, this:0x7fd501ff5530})
[2023-07-23 11:04:40.002395] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=167][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:40.002407] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2139) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF_DUMP](palf_id=1, self="192.168.1.1:1001", ack_info_list=[])
[2023-07-23 11:04:40.002416] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:40.002423] INFO  [PALF] report_log_task_trace (log_sliding_window.cpp:3999) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=6] current log_task status(palf_id=1, self="192.168.1.1:1001", log_id=32949108, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615}, max_scn:{val:18446744073709551615}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:0, submit_wait_time:0, submit_ts:0, flush_cost:0, flushed_ts:0}, ack_info_list=[])
[2023-07-23 11:04:40.014663] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=21] Cache replace map node details(ret=0, replace_node_count=0, replace_time=9329, replace_start_pos=22400000, replace_num=100000)
[2023-07-23 11:04:40.015908] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.015918] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081479960083], now0=[mts=1690081479960083], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.015928] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:130, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.015949] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=20][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:40.015961] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:130, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.015979] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=17] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479956872, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:130, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.016013] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=33][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.016021] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.016025] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.016031] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.016035] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:40.016042] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=127, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:40.016050] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7] will sleep(sleep_us=100000, remain_us=15153797, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=127, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:40.017451] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.028107] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=19][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:40.028119] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=13][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039b45418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196763}, scn:1690081479989519}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196763}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479917959, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.028151] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=32][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196763}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479917959, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.028305] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=154][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:40.028312] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.028317] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:40.028325] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.028329] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:40.028335] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:40.028339] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:40.028344] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702503991, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196763}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479917959, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039b4a628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196763}, scn:1690081479989519}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:40.028723] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=379][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:40.028728] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:40.028736] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:40.028740] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:40.028743] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:40.028747] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:40.028755] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:40.028758] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:40.028764] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:40.028768] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:40.028773] WDIAG [SQL] open (ob_result_set.cpp:159) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:40.028777] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:40.028800] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:64, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:40.028814] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196763}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479917959, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.028843] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196763}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479917959, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.028868] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081479989519, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196763}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479917959, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.028897] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:40.028904] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:40.028914] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:40.028921] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.028937] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081480027996 2023-07-23 03:04:40.027996|[process_begin] u=0 in_queue_time:6752756, receive_ts:1690081473275239, enqueue_ts:1690081480027986|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=4 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=50 |[check_priv] u=8 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=11 |[insert_start] u=1 |[get_das_id_begin] u=12 |[get_das_id_end] u=1 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=241 |[sync_auto_value] u=404 |[close_plan_begin] u=62 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=82 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=39 |[process_end] u=1 run_ts:1690081480028001|total_timeu=940)
[2023-07-23 11:04:40.029249] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079324-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.029716] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079324-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.030167] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079324-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.030414] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.030622] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079324-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.032480] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:91, local_retry_times:91, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.032514] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=25][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.032533] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=19][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.032545] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.032571] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.032666] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.038082] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C82-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1690081480038069})
[2023-07-23 11:04:40.038090] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC34-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.038108] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3291][][T0][YB420A01819C-0005FDE715FDDC34-0-0] [lt=16][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.041956] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=27] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1690081480041946})
[2023-07-23 11:04:40.041965] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC35-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.042984] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35E-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1690081480042978})
[2023-07-23 11:04:40.042990] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC36-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.043026] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=17] new mysql sessid created(conn.sessid_=3222102300, support_ssl=false)
[2023-07-23 11:04:40.043057] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=13] sm conn init succ(conn.sessid_=3222102300, sess.client_addr_="127.0.0.1:57976")
[2023-07-23 11:04:40.043069] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=8] accept one succ(*s={this:0x7fd3ef03a030, fd:360, err:0, last_decode_time_:0, last_write_time_:1690081480043056, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.043103] INFO  [SERVER] deliver_mysql_request (ob_srv_deliver.cpp:518) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=11] mysql login queue(mysql_queue_->queue_.size()=18446744073709551611)
[2023-07-23 11:04:40.043118] INFO  [SERVER] extract_user_tenant (obmp_connect.cpp:1243) [3194][MysqlQueueTh2][T0][Y0-0005FDE6FC4C4543-0-0] [lt=28] username and tenantname(user_name=ocp_monitor, tenant_name=)
[2023-07-23 11:04:40.043166] INFO  load_privilege_info (obmp_connect.cpp:426) [3194][MysqlQueueTh2][T1][Y0-0005FDE6FC4C4543-0-0] [lt=11] no tenant name set, use default tenant name(tenant_name=sys)
[2023-07-23 11:04:40.043453] INFO  [SERVER] process (obmp_connect.cpp:369) [3194][MysqlQueueTh2][T1][Y0-0005FDE6FC4C4543-0-0] [lt=6] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222102300, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:40.043627] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3769][T1_L0_G0][T1][YB420A01819C-0005FDE72ED825D4-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1690081480043620})
[2023-07-23 11:04:40.043633] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC37-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.043670] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.043682] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=13][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081572900748, now=[mts=1690081479987688], now0=[mts=1690081479987688], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.043693] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479871708, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:40.043722] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=25][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:1003}, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60})
[2023-07-23 11:04:40.043735] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=13][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=3222118872, *tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479871708, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:40.043761] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=23] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081479871708, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.043797] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=36][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:501113, schema_version:1689916332789704, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.043807] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=8][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.043814] WDIAG [SQL] open (ob_result_set.cpp:157) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.043819] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=4][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:43, local_retry_times:0, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:2, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.043828] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:82) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9][errcode=-4283] result set open failed, check if need retry(ret=-4283, cli_ret=-4283, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:40.043839] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:89) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] close result set fail(cret=-4283)
[2023-07-23 11:04:40.043847] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7][errcode=-4283] execute query fail(ret=-4283, timeout_timestamp=1690081572970748)
[2023-07-23 11:04:40.043854] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=5][errcode=-4283] try to execute again(ret=-4283, type=1, retry_type=2, timeout_remain=92926895)
[2023-07-23 11:04:40.043870] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081479987636 2023-07-23 03:04:39.987636|[process_begin] u=0 in_queue_time:7016888, receive_ts:1690081472970748, enqueue_ts:1690081479987626|[start_sql] u=1 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=32 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[pc_choose_plan] u=3 |[check_priv] u=3 |[plan_id] u=1 plan_id:45401|[do_open_plan_begin] u=2 plan_id:45401|[sql_start_stmt_begin] u=0 |[post_packet] u=18 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=56103 |[do_open_plan_end] u=7 |[close_plan_begin] u=22 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=3 |[query_end] u=29 |[process_end] u=1 run_ts:1690081479987638|total_timeu=56233)
[2023-07-23 11:04:40.045425] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.045442] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480045421]})
[2023-07-23 11:04:40.045456] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480045451]})
[2023-07-23 11:04:40.045491] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.045509] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=16][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.045530] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.045540] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=9][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.045614] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480045606]})
[2023-07-23 11:04:40.045714] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=12][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.045735] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=18][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.045771] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3382][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3222111617, support_ssl=false)
[2023-07-23 11:04:40.045766] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=11][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.045801] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3382][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=10] sm conn init succ(conn.sessid_=3222111617, sess.client_addr_="127.0.0.1:57978")
[2023-07-23 11:04:40.045811] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3382][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=6] accept one succ(*s={this:0x7fd4169c2030, fd:387, err:0, last_decode_time_:0, last_write_time_:1690081480045800, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.045800] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=34][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.045825] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=23] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.045855] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=26] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.045869] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05D-0-0] [lt=10] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480045865, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.045890] INFO  [SERVER] extract_user_tenant (obmp_connect.cpp:1243) [3196][MysqlQueueTh4][T0][Y0-0005FDE6FBCC4EF7-0-0] [lt=27] username and tenantname(user_name=ocp_monitor, tenant_name=)
[2023-07-23 11:04:40.045928] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F6-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.045988] INFO  load_privilege_info (obmp_connect.cpp:426) [3196][MysqlQueueTh4][T1][Y0-0005FDE6FBCC4EF7-0-0] [lt=27] no tenant name set, use default tenant name(tenant_name=sys)
[2023-07-23 11:04:40.046596] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C3F-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.046699] INFO  [SERVER] process (obmp_connect.cpp:369) [3196][MysqlQueueTh4][T1][Y0-0005FDE6FBCC4EF7-0-0] [lt=17] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222111617, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:40.046910] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3770][T1_L0_G0][T1][YB420A01819C-0005FDE72CD90295-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1690081480046904})
[2023-07-23 11:04:40.046917] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC38-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.048283] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC39-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.048279] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=22] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480048269})
[2023-07-23 11:04:40.049312] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.049498] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480048155 2023-07-23 03:04:40.048155|[process_begin] u=0 in_queue_time:4094834, receive_ts:1690081475953320, enqueue_ts:1690081480048142|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=4 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=88 |[plc_serialize_begin] u=3 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=6 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=50 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=3 |[pc_choose_plan] u=2 |[check_priv] u=5 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=4 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=12 |[do_open_plan_end] u=0 |[get_das_id_begin] u=81 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=26 |[calc_partition_location_begin] u=796 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=12 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[remote_task_completed] u=60 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=3 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=7 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=1 |[close_plan_begin] u=42 |[close_das_task_begin] u=1 |[close_das_task_end] u=40 |[start_end_stmt] u=35 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=24 |[process_end] u=1 run_ts:1690081480048160|total_timeu=1340)
[2023-07-23 11:04:40.051836] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=21][errcode=-4009] open file error(filename="cgroup/tenant_1/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:40.051867] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=32][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1/cpuacct.usage", usage_value="", tenant_id=1)
[2023-07-23 11:04:40.051974] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35E-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:40.052515] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [3769][T1_L0_G0][T1][YB420A01819C-0005FDE72ED825D4-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:40.052806] INFO  [SERVER] process (obmp_quit.h:55) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35F-0-0] [lt=14] quit
[2023-07-23 11:04:40.052809] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd3ef03a030, fd:360, err:5, last_decode_time_:1690081480052788, last_write_time_:1690081480052621, read_buffer_.get_consume_sz():654, get_pending_flag():1, get_trace_id():YB420A01819C-0005FDE72D09E35F-0-0})
[2023-07-23 11:04:40.052815] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:152) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35F-0-0] [lt=9][errcode=-4009] sock has error(ret=-4009)
[2023-07-23 11:04:40.052825] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:929) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35F-0-0] [lt=8][errcode=-4009] write response fail(ret=-4009)
[2023-07-23 11:04:40.052828] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=14] kill and revert session(conn.sessid_=3222102300, proxy_sessid=0, server_id=3, ret=0)
[2023-07-23 11:04:40.052831] WDIAG [SERVER] response (obmp_base.cpp:86) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35F-0-0] [lt=6][errcode=-4009] failed to flush_buffer(ret=-4009)
[2023-07-23 11:04:40.052836] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:48) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E35F-0-0] [lt=4][errcode=-4009] response rpc result fail(ret=-4009)
[2023-07-23 11:04:40.052835] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=7] can close safely, do destroy(*s={this:0x7fd3ef03a030, fd:360, err:5, last_decode_time_:1690081480052788, last_write_time_:1690081480052621, read_buffer_.get_consume_sz():659, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.052849] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=8] unlock session of tenant(conn.sessid_=3222102300, proxy_sessid=0, conn.tenant_id_=1)
[2023-07-23 11:04:40.052857] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3386][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=7] connection close(sessid=3222102300, proxy_sessid=0, tenant_id=1, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:40.052880] INFO  [SERVER] run (obmp_disconnect.cpp:78) [3768][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=6] free session successfully(sessid=3222102300)
[2023-07-23 11:04:40.053832] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1001][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=16][errcode=-4009] open file error(filename="cgroup/tenant_1001/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:40.053854] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1001][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=22][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1001/cpuacct.usage", usage_value="", tenant_id=1001)
[2023-07-23 11:04:40.054241] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=26] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080742373882309}, clog_base_lsn:{lsn:8388096000}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831059622276}, all_id_meta:{id_meta:[{limited_id:1690081474308899777, latest_log_ts:{val:1690081457643664628}}, {limited_id:834000001, latest_log_ts:{val:1690081371645352093}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:40.054289] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=48] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1001}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741701524066}, clog_base_lsn:{lsn:703996121088}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831251246577}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:40.054326] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=36] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1002}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741745971437}, clog_base_lsn:{lsn:406789103616}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831214220192}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:40.054354] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=26] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1003}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741730241082}, clog_base_lsn:{lsn:375249862656}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831245549869}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:1, proposal_id:54, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:40.056827] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1002][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=9][errcode=-4009] open file error(filename="cgroup/tenant_1002/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:40.056862] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1002][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=34][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1002/cpuacct.usage", usage_value="", tenant_id=1002)
[2023-07-23 11:04:40.057015] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3136][TimezoneMgr][T1][YB420A01819C-0005FDE6E8DE7E69-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.057046] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3136][TimezoneMgr][T1][YB420A01819C-0005FDE6E8DE7E69-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.057111] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.057148] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14595636, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.057161] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.058407] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194F-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.058422] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD194F-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.058478] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:264) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C83-0-0] [lt=23][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:40.059043] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=11] can close safely, do destroy(*s={this:0x7fd2684b4030, fd:390, err:5, last_decode_time_:1690081479058148, last_write_time_:1690081479057260, read_buffer_.get_consume_sz():1756, get_pending_flag():2, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.059065] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=11] unlock session of tenant(conn.sessid_=3222113392, proxy_sessid=0, conn.tenant_id_=1)
[2023-07-23 11:04:40.059071] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=6] connection close(sessid=3222113392, proxy_sessid=0, tenant_id=1, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:40.059125] INFO  [SERVER] run (obmp_disconnect.cpp:78) [3767][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=13] free session successfully(sessid=3222113392)
[2023-07-23 11:04:40.059580] INFO  [SERVER] process (obmp_quit.h:55) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C84-0-0] [lt=18] quit
[2023-07-23 11:04:40.059571] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd3ee1fe480, fd:381, err:5, last_decode_time_:1690081480059553, last_write_time_:1690081480058597, read_buffer_.get_consume_sz():1719, get_pending_flag():1, get_trace_id():YB420A01819C-0005FDE72D1C9C84-0-0})
[2023-07-23 11:04:40.059591] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:152) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C84-0-0] [lt=11][errcode=-4009] sock has error(ret=-4009)
[2023-07-23 11:04:40.059600] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:929) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C84-0-0] [lt=9][errcode=-4009] write response fail(ret=-4009)
[2023-07-23 11:04:40.059608] WDIAG [SERVER] response (obmp_base.cpp:86) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C84-0-0] [lt=7][errcode=-4009] failed to flush_buffer(ret=-4009)
[2023-07-23 11:04:40.059606] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=27] kill and revert session(conn.sessid_=3222122072, proxy_sessid=0, server_id=3, ret=0)
[2023-07-23 11:04:40.059618] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:48) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C84-0-0] [lt=9][errcode=-4009] response rpc result fail(ret=-4009)
[2023-07-23 11:04:40.059621] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=14] can close safely, do destroy(*s={this:0x7fd3ee1fe480, fd:381, err:5, last_decode_time_:1690081480059553, last_write_time_:1690081480058597, read_buffer_.get_consume_sz():1724, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.059650] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=18] unlock session of tenant(conn.sessid_=3222122072, proxy_sessid=0, conn.tenant_id_=1)
[2023-07-23 11:04:40.059664] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=14] connection close(sessid=3222122072, proxy_sessid=0, tenant_id=1, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:40.059691] INFO  [SERVER] run (obmp_disconnect.cpp:78) [3767][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=7] free session successfully(sessid=3222122072)
[2023-07-23 11:04:40.061662] INFO  [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1331) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=6] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1)
[2023-07-23 11:04:40.061677] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1022) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1, mem_hold=211812352, mem_limit=858993455, cache_obj_num=732, cache_node_num=729)
[2023-07-23 11:04:40.061687] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1039) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=8] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=211812352, mem_limit=858993455, cache_obj_num=732, cache_node_num=729)
[2023-07-23 11:04:40.061695] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2024) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=8] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:40.064918] INFO  [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1799) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] Dumping All Cache Objs(alloc_obj_list.count()=732, alloc_obj_list=[{obj_id:1, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109325}, {obj_id:2, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111572}, {obj_id:3, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:341289}, {obj_id:4, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235127}, {obj_id:5, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184447}, {obj_id:6, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229842}, {obj_id:7, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179161}, {obj_id:8, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123908}, {obj_id:9, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128433}, {obj_id:11, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116945}, {obj_id:13, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129603}, {obj_id:15, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117454}, {obj_id:16, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109248}, {obj_id:17, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129226}, {obj_id:18, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109760}, {obj_id:19, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122149}, {obj_id:20, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114037}, {obj_id:21, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158335}, {obj_id:22, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121145}, {obj_id:23, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128430}, {obj_id:26, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226632}, {obj_id:29, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115569}, {obj_id:37, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145358}, {obj_id:38, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165873}, {obj_id:40, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127310}, {obj_id:41, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146874}, {obj_id:42, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131649}, {obj_id:43, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131770}, {obj_id:44, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125898}, {obj_id:45, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145121}, {obj_id:46, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130841}, {obj_id:47, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131994}, {obj_id:48, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121918}, {obj_id:49, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119714}, {obj_id:50, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118468}, {obj_id:51, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132696}, {obj_id:52, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125866}, {obj_id:53, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121229}, {obj_id:54, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118644}, {obj_id:55, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121560}, {obj_id:56, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118676}, {obj_id:57, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121565}, {obj_id:58, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118621}, {obj_id:59, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124750}, {obj_id:60, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120121}, {obj_id:61, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116266}, {obj_id:62, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141345}, {obj_id:63, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116282}, {obj_id:64, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120980}, {obj_id:65, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116682}, {obj_id:66, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124583}, {obj_id:67, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116330}, {obj_id:68, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117450}, {obj_id:69, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239802}, {obj_id:70, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187352}, {obj_id:73, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223151}, {obj_id:74, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170701}, {obj_id:76, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113692}, {obj_id:78, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125220}, {obj_id:81, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116890}, {obj_id:82, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115894}, {obj_id:84, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111136}, {obj_id:112, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121899}, {obj_id:113, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159037}, {obj_id:114, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118845}, {obj_id:116, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88994}, {obj_id:117, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151029}, {obj_id:118, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117628}, {obj_id:119, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118807}, {obj_id:120, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88832}, {obj_id:121, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:124, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109685}, {obj_id:135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170252}, {obj_id:139, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157877}, {obj_id:152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133335}, {obj_id:155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113973}, {obj_id:156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195192}, {obj_id:157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119284}, {obj_id:161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109728}, {obj_id:162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108544}, {obj_id:164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115837}, {obj_id:165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111259}, {obj_id:167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137904}, {obj_id:171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118032}, {obj_id:172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117558}, {obj_id:173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:306152}, {obj_id:174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108552}, {obj_id:175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112192}, {obj_id:176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109808}, {obj_id:177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111256}, {obj_id:178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112154}, {obj_id:179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112278}, {obj_id:180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147332}, {obj_id:181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120509}, {obj_id:184, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112479}, {obj_id:185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118190}, {obj_id:186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225682}, {obj_id:187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128216}, {obj_id:189, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120694}, {obj_id:190, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152472}, {obj_id:192, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124689}, {obj_id:194, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121903}, {obj_id:195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107757}, {obj_id:197, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110196}, {obj_id:198, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106208}, {obj_id:200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126038}, {obj_id:201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120794}, {obj_id:202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112445}, {obj_id:203, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138373}, {obj_id:204, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124081}, {obj_id:206, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110192}, {obj_id:213, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:373654}, {obj_id:214, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:505828}, {obj_id:215, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113398}, {obj_id:220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131588}, {obj_id:221, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144772}, {obj_id:230, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107418}, {obj_id:232, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115072}, {obj_id:233, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:234, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:93529}, {obj_id:235, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123553}, {obj_id:236, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116626}, {obj_id:238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121199}, {obj_id:239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106928}, {obj_id:240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112090}, {obj_id:264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:90440}, {obj_id:266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135852}, {obj_id:267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134273}, {obj_id:269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145191}, {obj_id:273, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123608}, {obj_id:274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108417}, {obj_id:278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111783}, {obj_id:305, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121676}, {obj_id:306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141832}, {obj_id:311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129452}, {obj_id:312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115712}, {obj_id:313, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124673}, {obj_id:322, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88886}, {obj_id:324, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140374}, {obj_id:328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113091}, {obj_id:330, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159280}, {obj_id:331, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125300}, {obj_id:332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117330}, {obj_id:337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123557}, {obj_id:345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111769}, {obj_id:346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123643}, {obj_id:356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125880}, {obj_id:361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112752}, {obj_id:362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110232}, {obj_id:363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108568}, {obj_id:364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113753}, {obj_id:365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119640}, {obj_id:402, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:857901}, {obj_id:403, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:452223}, {obj_id:411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110518}, {obj_id:421, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121605}, {obj_id:422, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128289}, {obj_id:426, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128217}, {obj_id:427, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107111}, {obj_id:429, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108426}, {obj_id:437, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117618}, {obj_id:438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138464}, {obj_id:439, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129287}, {obj_id:440, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:105488}, {obj_id:443, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115530}, {obj_id:447, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197761}, {obj_id:450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199814}, {obj_id:452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207533}, {obj_id:474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133309}, {obj_id:522, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:375064}, {obj_id:524, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480101}, {obj_id:534, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117040}, {obj_id:580, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122952}, {obj_id:605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:345007}, {obj_id:660, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223440}, {obj_id:661, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115072}, {obj_id:665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118024}, {obj_id:666, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116400}, {obj_id:667, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114060}, {obj_id:802, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120968}, {obj_id:855, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121960}, {obj_id:862, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114399}, {obj_id:863, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118344}, {obj_id:869, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117119}, {obj_id:876, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150075}, {obj_id:877, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175815}, {obj_id:925, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195664}, {obj_id:933, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129181}, {obj_id:944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132503}, {obj_id:945, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137742}, {obj_id:946, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126636}, {obj_id:948, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120424}, {obj_id:955, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127224}, {obj_id:956, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116704}, {obj_id:957, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114720}, {obj_id:1014, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173857}, {obj_id:1015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132988}, {obj_id:1016, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129138}, {obj_id:1083, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124264}, {obj_id:1145, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130168}, {obj_id:1146, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122296}, {obj_id:1169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280049}, {obj_id:1266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88881}, {obj_id:1269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:11552}, {obj_id:1271, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:916091}, {obj_id:1274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:1275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107490}, {obj_id:1276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110398}, {obj_id:1278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124488}, {obj_id:1279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121664}, {obj_id:1283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106461}, {obj_id:1284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142870}, {obj_id:1285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175250}, {obj_id:1286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161389}, {obj_id:1287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137813}, {obj_id:1288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208605}, {obj_id:1289, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156033}, {obj_id:1290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159046}, {obj_id:1291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191322}, {obj_id:1297, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144114}, {obj_id:1298, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131804}, {obj_id:1299, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167182}, {obj_id:1300, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140586}, {obj_id:1301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237139}, {obj_id:1302, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162351}, {obj_id:1303, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220896}, {obj_id:1306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143476}, {obj_id:1307, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164310}, {obj_id:1308, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121121}, {obj_id:1309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220799}, {obj_id:1310, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142485}, {obj_id:1311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204086}, {obj_id:1315, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153942}, {obj_id:1316, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132418}, {obj_id:1317, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213169}, {obj_id:1318, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176390}, {obj_id:1319, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:428742}, {obj_id:1320, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208446}, {obj_id:1321, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:420997}, {obj_id:1325, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147492}, {obj_id:1326, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182923}, {obj_id:1327, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:811052}, {obj_id:1328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:300175}, {obj_id:1329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:967217}, {obj_id:1330, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286896}, {obj_id:1334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143185}, {obj_id:1335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131501}, {obj_id:1336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162857}, {obj_id:1337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:411417}, {obj_id:1338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216472}, {obj_id:1339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:488389}, {obj_id:1340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199461}, {obj_id:1345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145055}, {obj_id:1346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169934}, {obj_id:1347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126655}, {obj_id:1348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248008}, {obj_id:1349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149096}, {obj_id:1350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238697}, {obj_id:1353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132741}, {obj_id:1354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171395}, {obj_id:1355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251238}, {obj_id:1356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165610}, {obj_id:1359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145342}, {obj_id:1360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172824}, {obj_id:1361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257085}, {obj_id:1362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241945}, {obj_id:1367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146300}, {obj_id:1368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177171}, {obj_id:1369, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274985}, {obj_id:1370, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260698}, {obj_id:1372, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:466085}, {obj_id:1373, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:554863}, {obj_id:1380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132111}, {obj_id:1381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:148619}, {obj_id:1382, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175303}, {obj_id:1385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143827}, {obj_id:1386, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165757}, {obj_id:1387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132159}, {obj_id:1388, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226691}, {obj_id:1389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155636}, {obj_id:1390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210276}, {obj_id:1400, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153845}, {obj_id:1522, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132816}, {obj_id:1538, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:296281}, {obj_id:1641, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146892}, {obj_id:1642, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:1643, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283433}, {obj_id:1644, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275590}, {obj_id:1645, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142583}, {obj_id:1648, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143372}, {obj_id:1669, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:468828}, {obj_id:1686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152827}, {obj_id:1781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:380248}, {obj_id:1782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:517009}, {obj_id:1841, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117704}, {obj_id:2349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122600}, {obj_id:2362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111073}, {obj_id:2363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112556}, {obj_id:2365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140294}, {obj_id:2366, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133478}, {obj_id:2394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260616}, {obj_id:2431, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117352}, {obj_id:2451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121272}, {obj_id:2452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:2491, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151697}, {obj_id:2500, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135218}, {obj_id:2502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145948}, {obj_id:2503, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175710}, {obj_id:2504, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269768}, {obj_id:2505, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:255176}, {obj_id:2515, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180033}, {obj_id:2516, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182644}, {obj_id:2517, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:291295}, {obj_id:2518, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225595}, {obj_id:2519, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141982}, {obj_id:2535, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163630}, {obj_id:2536, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193037}, {obj_id:2538, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:558139}, {obj_id:2561, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156119}, {obj_id:2687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123272}, {obj_id:2712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:335251}, {obj_id:2864, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:357639}, {obj_id:2868, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:453965}, {obj_id:2924, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139696}, {obj_id:3119, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124568}, {obj_id:3201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134402}, {obj_id:3202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137472}, {obj_id:3203, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132350}, {obj_id:3275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138117}, {obj_id:3276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127600}, {obj_id:3277, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107922}, {obj_id:3279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110460}, {obj_id:3280, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116675}, {obj_id:3283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:3284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118296}, {obj_id:3285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137904}, {obj_id:3286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112502}, {obj_id:3288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113940}, {obj_id:3290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106432}, {obj_id:3291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:3293, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118680}, {obj_id:3294, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107219}, {obj_id:3295, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127909}, {obj_id:3309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:390507}, {obj_id:3473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130856}, {obj_id:3480, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151736}, {obj_id:3498, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319008}, {obj_id:3748, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138080}, {obj_id:3749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115376}, {obj_id:3938, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125896}, {obj_id:3993, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135764}, {obj_id:4060, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249784}, {obj_id:4061, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109801}, {obj_id:4062, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:4063, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:4064, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4065, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:4066, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4067, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88838}, {obj_id:4068, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4069, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4072, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4073, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88839}, {obj_id:4074, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4075, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4079, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133294}, {obj_id:4080, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:4081, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107168}, {obj_id:4084, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88964}, {obj_id:4195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:333559}, {obj_id:4200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:416289}, {obj_id:4267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131792}, {obj_id:4268, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116016}, {obj_id:4301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286580}, {obj_id:4518, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126536}, {obj_id:4546, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:4547, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4548, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4549, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88839}, {obj_id:4550, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4551, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112431}, {obj_id:4556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4568, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:299521}, {obj_id:4677, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173951}, {obj_id:4678, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202163}, {obj_id:4683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185458}, {obj_id:4684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219118}, {obj_id:4687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:792994}, {obj_id:4691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:805593}, {obj_id:4692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:960692}, {obj_id:4726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150855}, {obj_id:4866, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88952}, {obj_id:5048, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:259342}, {obj_id:5051, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:302866}, {obj_id:5079, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125592}, {obj_id:5128, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136112}, {obj_id:5287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:340096}, {obj_id:5291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:426188}, {obj_id:5364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127560}, {obj_id:5380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120632}, {obj_id:5381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124904}, {obj_id:5559, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:359785}, {obj_id:5561, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:456815}, {obj_id:5615, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257334}, {obj_id:5651, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136416}, {obj_id:5652, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174265}, {obj_id:5653, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129528}, {obj_id:5884, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234650}, {obj_id:5909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177506}, {obj_id:6176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142016}, {obj_id:6177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151189}, {obj_id:6178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120328}, {obj_id:6208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126232}, {obj_id:6385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274624}, {obj_id:6387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:329196}, {obj_id:6470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171259}, {obj_id:6578, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190649}, {obj_id:6579, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215440}, {obj_id:6584, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193411}, {obj_id:6590, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205876}, {obj_id:6631, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156453}, {obj_id:6774, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174901}, {obj_id:7027, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171911}, {obj_id:7060, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263837}, {obj_id:7191, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:344496}, {obj_id:7195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:433143}, {obj_id:7208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121656}, {obj_id:7263, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132480}, {obj_id:7264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150203}, {obj_id:7450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:278957}, {obj_id:7452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:332196}, {obj_id:7459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250592}, {obj_id:7462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290128}, {obj_id:7486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126920}, {obj_id:7547, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170957}, {obj_id:7806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:748217}, {obj_id:7807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:889404}, {obj_id:7813, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:904154}, {obj_id:7814, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1074659}, {obj_id:7882, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162719}, {obj_id:7921, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283317}, {obj_id:8073, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272428}, {obj_id:8075, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:322991}, {obj_id:8143, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141376}, {obj_id:8144, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145305}, {obj_id:8367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:346670}, {obj_id:8368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:436340}, {obj_id:8449, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167665}, {obj_id:8450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119976}, {obj_id:8483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276813}, {obj_id:8675, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270240}, {obj_id:8676, tenant_id:1, log_del_time:9223372036854775807, real_de
[2023-07-23 11:04:40.065891] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2032) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=972] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:40.078851] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B84-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.078866] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B84-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.079264] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.079433] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480078439 2023-07-23 03:04:40.078439|[process_begin] u=0 in_queue_time:4054612, receive_ts:1690081476023826, enqueue_ts:1690081480078422|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=45 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=5 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=4 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=5 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=25 |[calc_partition_location_begin] u=669 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=9 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=50 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=17 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=1 |[close_plan_begin] u=73 |[close_das_task_begin] u=1 |[close_das_task_end] u=34 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[query_end] u=12 |[process_end] u=1 run_ts:1690081480078442|total_timeu=992)
[2023-07-23 11:04:40.080184] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [83145][T1002_LSArchive][T1002][YB420A01819C-0005FDE628078DB5-0-0] [lt=18] gc stale ls task succ
[2023-07-23 11:04:40.088754] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=15][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.088765] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081480032651], now0=[mts=1690081480032651], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.088776] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908679}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480031648, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.088781] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196764}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479977923, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.088806] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=27][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:40.088819] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908679}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480031648, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.088820] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=39] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196764}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081479977923, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.088840] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=21] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081479989521, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908679}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480031648, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.088856] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=34] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480045271, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196764}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081479977923, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.088878] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=38][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.088888] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.088895] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.088899] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.088903] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:40.088910] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=92, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:40.088899] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480088592 2023-07-23 03:04:40.088592|[process_begin] u=0 in_queue_time:7428456, receive_ts:1690081472660135, enqueue_ts:1690081480088575|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=63 |[check_priv] u=13 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=3 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=6 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[insert_start] u=19 |[get_das_id_begin] u=21 |[get_das_id_end] u=0 |[insert_end] u=5 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=13 |[sync_auto_value] u=5 |[close_plan_begin] u=19 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=106 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=12 |[process_end] u=0 run_ts:1690081480088595|total_timeu=305)
[2023-07-23 11:04:40.088919] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8] will sleep(sleep_us=92000, remain_us=20587619, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=92, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:40.089602] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F6-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.090448] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F6-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.091257] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F6-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.091622] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:309) [3730][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=6] ====== traversal_flush timer task ======
[2023-07-23 11:04:40.091671] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:341) [3730][T1_Flush][T1][Y0-0000000000000000-0-0] [lt=19] succeed to traversal_flush(ret=0, ls_cnt=1)
[2023-07-23 11:04:40.092072] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F6-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.092609] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.092627] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.092639] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14577347, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.092653] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14686032, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.092651] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.092670] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get wrs ts(ls_id={id:1001}, delta=14685794, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.092666] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.092679] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.092693] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1002}, delta=15015081, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.092701] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.092714] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1003}, delta=14949222, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.092722] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.098844] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.098935] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480098668 2023-07-23 03:04:40.098668|[process_begin] u=0 in_queue_time:4873977, receive_ts:1690081475224690, enqueue_ts:1690081480098650|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=10 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=66 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=4 |[get_plan_type_end] u=2 |[pc_choose_plan] u=3 |[check_priv] u=5 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=3 plan_id:17927|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=4 |[parse_job_end] u=7 |[remote_task_completed] u=93 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29253}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=25 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=3 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=5 |[query_end] u=21 |[process_end] u=1 run_ts:1690081480098679|total_timeu=264)
[2023-07-23 11:04:40.105399] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [83382][T1001_LSArchive][T1001][YB420A01819C-0005FDE628A78DB5-0-0] [lt=8] gc stale ls task succ
[2023-07-23 11:04:40.106787] INFO  [OCCAM] ObOccamTimerTaskRAIIHandle (ob_occam_timer.h:363) [83036][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=58] task handle constructed(*this={this:0x7fd4b094b9e0, task:NULL, is_inited:false, is_running:false})
[2023-07-23 11:04:40.107352] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3308][Occam][T1][YB420A01819C-0005FDE625B7976A-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.107383] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:644) [3308][Occam][T1][YB420A01819C-0005FDE625B7976A-0-0] [lt=0] inner session no need register to deadlock(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4122, exec_ctx.get_errcode()="OB_RPC_POST_ERROR", session={this:0x7fd4350980d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fd7290}, desc=NULL, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.107400] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [3308][Occam][T1][YB420A01819C-0005FDE625B7976A-0-0] [lt=17] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4122, exec_ctx.get_errcode()="OB_RPC_POST_ERROR", session={this:0x7fd4350980d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fd7290}, desc=NULL, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.107413] INFO  [STORAGE.TRANS] rollback_to_global_implicit_savepoint_ (ob_tx_api.cpp:1023) [3308][Occam][T1][YB420A01819C-0005FDE625B7976A-0-0] [lt=11] add tx part with extra_touched_ls(extra_touched_ls=[{id:1}], tx.tx_id={txid:44908680})
[2023-07-23 11:04:40.107432] INFO  [STORAGE.TRANS] abort_tx_ (ob_trans_service_v4.cpp:575) [3308][Occam][T1][YB420A01819C-0005FDE625B7976A-0-0] [lt=0] abort tx(ret=0, *this={is_inited_:true, tenant_id_:1, this:0x7fd550e04030}, tx={this:0x7fd4f4fd7290, tx_id:{txid:44908680}, state:6, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081480106234, op_sn:3, alloc_ts:1690081480107274, active_ts:1690081480107274, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081510107267, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"0.0.0.0:0", epoch:-1, first_scn:9223372036854775807, last_scn:9223372036854775807, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:4, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, cause=4)
[2023-07-23 11:04:40.107461] INFO  [STORAGE.TRANS] start_epoch_ (ob_tx_api.cpp:1736) [3308][Occam][T1][YB420A01819C-0005FDE625B7976A-0-0] [lt=29] tx start new epoch(ret=0, tx={this:0x7fd4f4fd7290, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:0, op_sn:4, alloc_ts:1690081480107274, active_ts:0, commit_ts:0, finish_ts:0, timeout_us:0, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:40.107485] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [3308][Occam][T1][YB420A01819C-0005FDE625B7976A-0-0] [lt=21] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480106234, tx_desc={this:0x7fd4f4fd7290, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:0, op_sn:4, alloc_ts:1690081480107274, active_ts:0, commit_ts:0, finish_ts:0, timeout_us:0, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[{id:1}], cflict_txs:[]}, rollback=true, session={this:0x7fd4350980d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fd7290}, exec_ctx.get_errcode()=-4122)
[2023-07-23 11:04:40.107525] INFO  [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1915) [3308][Occam][T0][Y0-0000000000000000-0-0] [lt=0] execute write sql(ret=-4122, tenant_id=1, affected_rows=0, sql=INSERT INTO __all_server_event_history (gmt_create, module, event, name1, value1, name2, value2, name3, value3, name4, value4, svr_ip, svr_port) VALUES (usec_to_time(1690081480106829), 'ELECTION', 'vote', 'TENANT_ID', 1002, 'LS_ID', 1002, 'INFO', 'vote for "192.168.1.1:1001", reason:the only request', 'HAPPENED_TIME', '2023-07-23 11:04:40.106782', '192.168.1.1', 2882))
[2023-07-23 11:04:40.113396] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.113411] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.113419] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.113433] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.113439] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=5][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.113609] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.113624] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=14][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.113638] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=7][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.113663] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=23][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.113682] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=17] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.113702] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=17] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.113715] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=7] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480113712, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.114194] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3284][AutoLSLoc][T1][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.114681] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3284][AutoLSLoc][T1001][YB420A01819C-0005FDE7200AAFA8-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.116115] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:127, local_retry_times:127, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.116129] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.116133] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.116138] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.116146] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.116199] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.118502] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD7-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.118517] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD7-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.130469] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.139259] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196765}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480028486, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.139304] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=45] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196765}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480028486, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.139330] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480113486, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196765}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480028486, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.139372] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480139099 2023-07-23 03:04:40.139099|[process_begin] u=0 in_queue_time:6863859, receive_ts:1690081473275239, enqueue_ts:1690081480139085|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=50 |[check_priv] u=9 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=3 |[value_after_calc_row] u=11 |[insert_start] u=1 |[get_das_id_begin] u=13 |[get_das_id_end] u=1 |[insert_end] u=2 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=35 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=100 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=11 |[process_end] u=1 run_ts:1690081480139102|total_timeu=272)
[2023-07-23 11:04:40.139697] WDIAG [RPC.FRAME] create_session (ob_req_transport.cpp:263) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079325-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.139713] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079325-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.139870] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879360-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.139892] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879360-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.140180] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079325-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.140637] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079325-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.145956] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480145945]})
[2023-07-23 11:04:40.145976] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=1] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480145971]})
[2023-07-23 11:04:40.145982] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.145991] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.146004] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.146010] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.146126] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480146119]})
[2023-07-23 11:04:40.146163] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.146173] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.146185] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=7][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.146207] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=22][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.146225] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=17] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.146243] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=17] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.146255] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05E-0-0] [lt=9] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480146253, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.146440] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F7-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.146849] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF3-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.149265] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC3A-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.149261] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=73] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480149252})
[2023-07-23 11:04:40.150056] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.150195] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480149167 2023-07-23 03:04:40.149167|[process_begin] u=0 in_queue_time:4195846, receive_ts:1690081475953320, enqueue_ts:1690081480149155|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=65 |[plc_serialize_begin] u=3 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=5 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=31 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=2 |[check_priv] u=12 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=18 |[do_open_plan_end] u=0 |[get_das_id_begin] u=39 |[get_das_id_end] u=1 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=22 |[calc_partition_location_begin] u=631 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=10 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=42 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=6 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=31 |[close_das_task_begin] u=2 |[close_das_task_end] u=31 |[start_end_stmt] u=22 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=17 |[process_end] u=1 run_ts:1690081480149170|total_timeu=1026)
[2023-07-23 11:04:40.157143] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.157172] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14695513, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.157184] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.159263] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1950-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.159389] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.167925] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.167943] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.167954] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.167964] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.167978] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.167988] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.167996] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168005] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168014] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168024] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168033] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168043] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168052] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168062] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168073] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.168083] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.171987] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.171999] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=13][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081480116185], now0=[mts=1690081480116185], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.172010] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:131, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.172036] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=25][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:40.172051] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=1] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480113485, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:131, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.172096] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] will sleep(sleep_us=100000, remain_us=14997752, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=128, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:40.180046] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.180126] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480179500 2023-07-23 03:04:40.179500|[process_begin] u=0 in_queue_time:4155673, receive_ts:1690081476023826, enqueue_ts:1690081480179485|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=31 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=16 |[calc_partition_location_begin] u=444 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=29 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=22 |[close_das_task_begin] u=0 |[close_das_task_end] u=25 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=8 |[process_end] u=0 run_ts:1690081480179503|total_timeu=624)
[2023-07-23 11:04:40.180983] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.181061] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.189791] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196766}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480088357, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.189828] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=37] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196766}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480088357, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.189855] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480145106, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196766}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480088357, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.189897] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480189639 2023-07-23 03:04:40.189639|[process_begin] u=0 in_queue_time:7529503, receive_ts:1690081472660135, enqueue_ts:1690081480189626|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=56 |[check_priv] u=9 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=1 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=3 |[value_after_calc_row] u=12 |[insert_start] u=1 |[get_das_id_begin] u=13 |[get_das_id_end] u=1 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=10 |[sync_auto_value] u=2 |[close_plan_begin] u=20 |[insert_close] u=0 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=93 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=11 |[process_end] u=1 run_ts:1690081480189642|total_timeu=257)
[2023-07-23 11:04:40.190300] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879361-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.190813] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879361-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.191319] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879361-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.191814] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879361-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.192665] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.192676] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.192683] INFO  [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:808) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=false, can_update_version=false, cur_leader_epoch=0, start_service_tstamp_=0, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0)
[2023-07-23 11:04:40.192693] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14785970, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.192700] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=5] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.192706] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get wrs ts(ls_id={id:1}, delta=14677285, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.192714] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.192731] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get wrs ts(ls_id={id:1001}, delta=14785732, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.192739] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.192752] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1002}, delta=15115019, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.192760] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.192773] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1003}, delta=15049160, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.192781] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.194369] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73136676044, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:40.195547] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash time detail, (compute_wash_size_time=98, refresh_score_time=682, wash_time=477)
[2023-07-23 11:04:40.199743] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=45][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480146253})
[2023-07-23 11:04:40.199763] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=20][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480146253}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.199782] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.199789] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.199797] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480146253}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:40.199822] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=25][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:40.199833] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:40.199838] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:40.199845] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:40.201723] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:40.201778] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=21][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480146253})
[2023-07-23 11:04:40.201800] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=22][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480146253}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.201818] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.201824] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.201830] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.201838] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:40.201843] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:40.201850] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.201858] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.201862] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.201868] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:40.201876] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:40.201880] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:40.201886] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:40.201894] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:40.201901] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:40.201908] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:40.201913] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:40.201922] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:50, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:40.201932] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.201942] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=5022749)
[2023-07-23 11:04:40.201969] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949073673, cache_obj->added_lc()=false, cache_obj->get_object_id()=20501, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.201994] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10] [packet retry query](TRACE=begin_ts=1690081480199707 2023-07-23 03:04:40.199707|[process_begin] u=0 in_queue_time:4975017, receive_ts:1690081475224690, enqueue_ts:1690081480199697|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=3 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=26 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=103 |[transform_with_outline_begin] u=91 |[transform_with_outline_end] u=1 |[resolve_begin] u=4 |[resolve_end] u=638 |[transform_begin] u=28 |[optimize_begin] u=871 |[tl_calc_part_id_end] u=233 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=212 |[query_end] u=70 |[process_end] u=1 run_ts:1690081480199711|total_timeu=2286)
[2023-07-23 11:04:40.202024] INFO  do_work (ob_rl_mgr.cpp:704) [3225][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=25] swc wakeup.(stat_period_=1000000, ready=false)
[2023-07-23 11:04:40.202430] INFO  [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1331) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=7] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1002)
[2023-07-23 11:04:40.202444] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1022) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1002, mem_hold=869539840, mem_limit=1063004405, cache_obj_num=1984, cache_node_num=1958)
[2023-07-23 11:04:40.202454] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1039) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=8] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=869539840, mem_limit=1063004405, cache_obj_num=1984, cache_node_num=1958)
[2023-07-23 11:04:40.202463] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2024) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=9] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:40.202289] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2135) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=19] [PALF_DUMP](palf_id=1002, self="192.168.1.1:1001", [SlidingWindow]={palf_id:1002, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:130559082, max_lsn:406891815781, max_scn:{val:1690081465085443984}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:406891815781}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:406891815358}, last_submit_end_lsn:{lsn:406891815781}, last_submit_log_id:130559082, last_submit_log_pid:1, max_flushed_lsn:{lsn:406891815358}, max_flushed_end_lsn:{lsn:406891815781}, max_flushed_log_pid:1, committed_end_lsn:{lsn:406891815358}, last_slide_log_id:130559081, last_slide_scn:{val:1690081465077463222}, last_slide_lsn:{lsn:406891814580}, last_slide_end_lsn:{lsn:406891815358}, last_slide_log_pid:1, last_slide_log_accum_checksum:2796067235, last_fetch_end_lsn:{lsn:406933758398}, last_fetch_max_log_id:130561129, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd45c242530}, [StateMgr]={this:0x7fd45c2453f0, self:"192.168.1.1:1001", palf_id:1002, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"0.0.0.0:0", leader_epoch:0, is_sync_enabled:true, allow_vote:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:130559082, is_changing_config_with_arb:false, reconfirm_start_time_us:-1, palf_role_change_cb:0x7fd45c24b670, allow_vote:true}, [ConfigMgr]={palf_id:1002, self:"192.168.1.1:1001", alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, prev_log_proposal_id:9223372036854775807, applied_alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, applied_alive_paxos_replica_num:3, applied_all_learnerlist:{learner_num:0, learner_array:[]}, prev_lsn:{lsn:18446744073709551615}, prev_mode_pid:9223372036854775807, state:0, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, region:"default_region", paxos_member_region_map:{}, register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7fd45c2435b0}, [ModeMgr]={palf_id:1002, self:"192.168.1.1:1001", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628808843420}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628808843420}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628808843420}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807}, [LogEngine]={palf_id:1002, is_inited:true, min_block_max_scn:{val:1689906401725815657}, min_block_id:5931, base_lsn_for_block_gc:{lsn:406789103616}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:1, base_lsn:{lsn:406789103616}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628808843420}}}, log_meta_storage:{log_tail:{lsn:1134592}, readable_log_tail:{lsn:1134592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:406923313152}, min_scn:{val:0}, curr_block_id:0, palf_id:1002}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1002/meta", dir_fd:481, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65970176, block_header_serialize_buf_:0x7fd45c247030}, log_storage:{log_tail:{lsn:406891815781}, readable_log_tail:{lsn:406891815781}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:406856208384}, min_scn:{val:1690080934097393577}, curr_block_id:6063, palf_id:1002}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1002/log", dir_fd:482, min_block_id:5931, max_block_id:6064, curr_writable_block_id:6063}, logical_block_size_:67104768, curr_block_writable_size_:31497371, block_header_serialize_buf_:0x7fd45c248cb0}, palf_epoch:3, this:0x7fd45c245af0}, [Reconfirm]={palf_id:1002, self:"192.168.1.1:1001", state:"INITED", new_proposal_id:9223372036854775807, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:0, majority_max_log_server:"0.0.0.0:0", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:18446744073709551615}, saved_end_lsn:{lsn:18446744073709551615}, last_submit_prepare_req_time_us:-1, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, has_notify_fetch:false, this:0x7fd45c245530})
[2023-07-23 11:04:40.202513] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=220][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1002, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:40.202528] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2139) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=15] [PALF_DUMP](palf_id=1002, self="192.168.1.1:1001", ack_info_list=[])
[2023-07-23 11:04:40.202540] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1002, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:40.202551] INFO  [PALF] report_log_task_trace (log_sliding_window.cpp:3999) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=10] current log_task status(palf_id=1002, self="192.168.1.1:1001", log_id=130559082, log_task={header:{begin_lsn:{lsn:406891815358}, end_lsn:{lsn:18446744073709551615}, log_id:130559082, min_scn:{val:1690081465085443984}, max_scn:{val:1690081465085443984}, data_len:367, proposal_id:1, prev_lsn:{lsn:406891814580}, prev_proposal_id:1, committed_end_lsn:{lsn:406891815358}, data_checksum:987981821, accum_checksum:3827705464, is_padding_log:false, is_raw_write:false}, state_map:{val:26}, ref_cnt:0, gen_ts:1690081465085963, submit_wait_time:2, submit_ts:1690081465085965, flush_cost:895, flushed_ts:1690081465086860}, ack_info_list=[])
[2023-07-23 11:04:40.206315] INFO  [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1799) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] Dumping All Cache Objs(alloc_obj_list.count()=1984, alloc_obj_list=[{obj_id:59, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109179}, {obj_id:436, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241752}, {obj_id:549, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113540}, {obj_id:560, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125156}, {obj_id:564, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110196}, {obj_id:565, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108544}, {obj_id:691, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112090}, {obj_id:692, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112210}, {obj_id:1367, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:1419, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:4212, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:4220, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:5192, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109728}, {obj_id:11057, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108552}, {obj_id:11059, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109808}, {obj_id:11060, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111256}, {obj_id:11194, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88952}, {obj_id:11319, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:11320, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:15478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140587}, {obj_id:15745, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126810}, {obj_id:15746, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140248}, {obj_id:15747, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128659}, {obj_id:27929, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142362}, {obj_id:27931, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128737}, {obj_id:27971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143089}, {obj_id:28026, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122421}, {obj_id:28581, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107590}, {obj_id:33091, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128554}, {obj_id:34554, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122615}, {obj_id:35697, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116836}, {obj_id:35720, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118097}, {obj_id:35721, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89647}, {obj_id:35722, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89453}, {obj_id:35811, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129607}, {obj_id:35814, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117342}, {obj_id:35888, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141527}, {obj_id:36341, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115569}, {obj_id:36991, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118097}, {obj_id:36992, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89647}, {obj_id:36993, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:89453}, {obj_id:37327, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134320}, {obj_id:37578, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132608}, {obj_id:39068, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143906}, {obj_id:39453, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107024}, {obj_id:39469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115590}, {obj_id:39684, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115949}, {obj_id:39841, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111363}, {obj_id:39878, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112304}, {obj_id:39926, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115580}, {obj_id:39930, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118242}, {obj_id:40799, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111572}, {obj_id:40807, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138778}, {obj_id:40808, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128342}, {obj_id:40809, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122922}, {obj_id:40810, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110504}, {obj_id:41224, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141493}, {obj_id:41415, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88881}, {obj_id:41418, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:11552}, {obj_id:41419, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:916091}, {obj_id:41420, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106461}, {obj_id:41421, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134402}, {obj_id:41422, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137472}, {obj_id:41423, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132350}, {obj_id:41425, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109997}, {obj_id:42005, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119470}, {obj_id:42006, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140970}, {obj_id:42161, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243831}, {obj_id:42376, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121389}, {obj_id:42389, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231388}, {obj_id:42394, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243933}, {obj_id:42456, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237719}, {obj_id:42488, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242064}, {obj_id:42509, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227384}, {obj_id:42511, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193344}, {obj_id:42526, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136549}, {obj_id:42545, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191472}, {obj_id:42572, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143043}, {obj_id:42580, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237596}, {obj_id:42603, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147035}, {obj_id:42608, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239548}, {obj_id:42614, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235468}, {obj_id:42615, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226982}, {obj_id:42616, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239696}, {obj_id:42650, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235502}, {obj_id:42652, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235294}, {obj_id:42655, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195588}, {obj_id:42664, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250450}, {obj_id:42674, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233474}, {obj_id:42690, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136327}, {obj_id:42691, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141008}, {obj_id:42692, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237407}, {obj_id:42736, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235468}, {obj_id:42740, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189382}, {obj_id:42744, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176369}, {obj_id:42746, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122937}, {obj_id:42752, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235638}, {obj_id:42753, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229499}, {obj_id:42757, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233788}, {obj_id:42759, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231541}, {obj_id:42760, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235752}, {obj_id:42762, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239707}, {obj_id:42765, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233641}, {obj_id:42769, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233641}, {obj_id:42770, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231370}, {obj_id:42772, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225129}, {obj_id:42773, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227248}, {obj_id:42780, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229515}, {obj_id:42783, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229331}, {obj_id:42785, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237678}, {obj_id:42791, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225610}, {obj_id:42823, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237576}, {obj_id:42847, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139794}, {obj_id:42848, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139842}, {obj_id:42869, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141707}, {obj_id:42877, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233515}, {obj_id:42929, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196382}, {obj_id:42930, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166832}, {obj_id:42933, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200801}, {obj_id:42934, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168438}, {obj_id:42975, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182017}, {obj_id:42976, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158792}, {obj_id:43125, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192454}, {obj_id:43126, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163561}, {obj_id:43135, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200583}, {obj_id:43136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168860}, {obj_id:43145, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198468}, {obj_id:43146, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167145}, {obj_id:43165, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198417}, {obj_id:43166, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167142}, {obj_id:43185, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225027}, {obj_id:43187, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194300}, {obj_id:43188, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165170}, {obj_id:43209, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123779}, {obj_id:43210, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115125}, {obj_id:43217, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231456}, {obj_id:43235, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250319}, {obj_id:43251, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192358}, {obj_id:43252, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164022}, {obj_id:43289, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192317}, {obj_id:43290, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163512}, {obj_id:43307, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231386}, {obj_id:43341, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190036}, {obj_id:43342, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162726}, {obj_id:43343, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188060}, {obj_id:43344, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161506}, {obj_id:43350, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175697}, {obj_id:43351, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:154660}, {obj_id:43371, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223010}, {obj_id:43374, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231405}, {obj_id:43412, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190369}, {obj_id:43413, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161908}, {obj_id:43440, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178268}, {obj_id:43468, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233465}, {obj_id:43469, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185944}, {obj_id:43470, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160252}, {obj_id:43471, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183905}, {obj_id:43472, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159514}, {obj_id:43475, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192081}, {obj_id:43476, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163941}, {obj_id:43477, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194553}, {obj_id:43478, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163871}, {obj_id:43495, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186249}, {obj_id:43496, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160359}, {obj_id:43497, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190285}, {obj_id:43498, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162767}, {obj_id:43499, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194439}, {obj_id:43500, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164325}, {obj_id:43516, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235574}, {obj_id:43523, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235478}, {obj_id:43537, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198599}, {obj_id:43538, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166292}, {obj_id:43557, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198496}, {obj_id:43558, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167189}, {obj_id:43564, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179627}, {obj_id:43565, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157513}, {obj_id:43568, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200715}, {obj_id:43569, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168000}, {obj_id:43574, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190136}, {obj_id:43575, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162730}, {obj_id:43576, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182109}, {obj_id:43577, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157887}, {obj_id:43582, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231365}, {obj_id:43583, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198204}, {obj_id:43584, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167548}, {obj_id:43589, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241715}, {obj_id:43593, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200800}, {obj_id:43594, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167565}, {obj_id:43605, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194343}, {obj_id:43606, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164741}, {obj_id:43607, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180048}, {obj_id:43608, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157152}, {obj_id:43613, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227101}, {obj_id:43621, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195746}, {obj_id:43625, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196652}, {obj_id:43626, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165566}, {obj_id:43629, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222865}, {obj_id:43631, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237599}, {obj_id:43636, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250302}, {obj_id:43639, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190288}, {obj_id:43640, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162779}, {obj_id:43641, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145601}, {obj_id:43642, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138368}, {obj_id:43657, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:246169}, {obj_id:43660, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188278}, {obj_id:43661, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160674}, {obj_id:43666, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222992}, {obj_id:43667, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225091}, {obj_id:43680, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237833}, {obj_id:43683, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196413}, {obj_id:43684, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166866}, {obj_id:43690, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192452}, {obj_id:43691, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163159}, {obj_id:43692, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200627}, {obj_id:43693, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167944}, {obj_id:43697, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188317}, {obj_id:43698, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161594}, {obj_id:43700, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243805}, {obj_id:43707, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184148}, {obj_id:43708, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159121}, {obj_id:43711, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200664}, {obj_id:43712, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168429}, {obj_id:43713, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241821}, {obj_id:43714, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190251}, {obj_id:43715, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161845}, {obj_id:43716, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202618}, {obj_id:43717, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169167}, {obj_id:43720, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194445}, {obj_id:43721, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164779}, {obj_id:43741, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188208}, {obj_id:43742, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161547}, {obj_id:43768, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190252}, {obj_id:43769, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162775}, {obj_id:43772, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200668}, {obj_id:43773, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:168433}, {obj_id:43777, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234715}, {obj_id:43778, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183042}, {obj_id:43782, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231457}, {obj_id:43785, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196511}, {obj_id:43786, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165969}, {obj_id:43797, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235304}, {obj_id:43804, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190125}, {obj_id:43805, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162719}, {obj_id:43806, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198492}, {obj_id:43807, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167609}, {obj_id:43810, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196667}, {obj_id:43811, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166013}, {obj_id:43812, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192194}, {obj_id:43813, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163970}, {obj_id:43818, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194461}, {obj_id:43819, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165219}, {obj_id:43827, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186092}, {obj_id:43828, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160302}, {obj_id:43834, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186129}, {obj_id:43835, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160353}, {obj_id:43836, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180010}, {obj_id:43837, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157147}, {obj_id:43845, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179973}, {obj_id:43846, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156702}, {obj_id:43859, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181802}, {obj_id:43860, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157826}, {obj_id:43865, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227091}, {obj_id:43869, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158361}, {obj_id:43870, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150274}, {obj_id:43871, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231334}, {obj_id:43875, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227227}, {obj_id:43880, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198489}, {obj_id:43881, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167606}, {obj_id:43886, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198419}, {obj_id:43887, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167144}, {obj_id:43896, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188263}, {obj_id:43897, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160675}, {obj_id:43901, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196597}, {obj_id:43902, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165559}, {obj_id:43904, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192380}, {obj_id:43905, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164036}, {obj_id:43906, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198457}, {obj_id:43907, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166694}, {obj_id:43910, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196512}, {obj_id:43911, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165970}, {obj_id:43912, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190172}, {obj_id:43913, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163239}, {obj_id:43916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198420}, {obj_id:43917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167145}, {obj_id:43918, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191525}, {obj_id:43919, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190153}, {obj_id:43920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162283}, {obj_id:43921, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196379}, {obj_id:43922, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166376}, {obj_id:43923, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183894}, {obj_id:43924, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159068}, {obj_id:43925, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192355}, {obj_id:43926, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164019}, {obj_id:43927, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181986}, {obj_id:43928, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158336}, {obj_id:43929, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155366}, {obj_id:43930, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149136}, {obj_id:43931, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150600}, {obj_id:43933, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194326}, {obj_id:43934, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165167}, {obj_id:43935, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242084}, {obj_id:43937, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239847}, {obj_id:43939, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184120}, {obj_id:43940, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158668}, {obj_id:43941, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188386}, {obj_id:43942, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160694}, {obj_id:43943, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241982}, {obj_id:43945, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192218}, {obj_id:43946, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163510}, {obj_id:43950, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188212}, {obj_id:43951, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161552}, {obj_id:43952, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182092}, {obj_id:43953, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157925}, {obj_id:43955, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190310}, {obj_id:43956, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162328}, {obj_id:43959, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177852}, {obj_id:43960, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156382}, {obj_id:43961, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184029}, {obj_id:43962, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159081}, {obj_id:43963, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186068}, {obj_id:43964, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159851}, {obj_id:43966, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237619}, {obj_id:43967, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190235}, {obj_id:43968, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161845}, {obj_id:43969, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229386}, {obj_id:43970, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180013}, {obj_id:43971, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157150}, {obj_id:43972, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190188}, {obj_id:43973, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162766}, {obj_id:43974, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190379}, {obj_id:43975, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162374}, {obj_id:43977, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181994}, {obj_id:43978, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158341}, {obj_id:43980, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200528}, {obj_id:43981, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167941}, {obj_id:43982, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194333}, {obj_id:43983, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165174}, {obj_id:43985, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186241}, {obj_id:43986, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160356}, {obj_id:43988, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192305}, {obj_id:43989, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163100}, {obj_id:43991, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239789}, {obj_id:43992, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235419}, {obj_id:43993, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188226}, {obj_id:43994, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161997}, {obj_id:43996, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184079}, {obj_id:43997, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159571}, {obj_id:43999, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190148}, {obj_id:44000, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162726}, {obj_id:44001, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186031}, {obj_id:44002, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160766}, {obj_id:44003, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190204}, {obj_id:44004, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163198}, {obj_id:44006, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194309}, {obj_id:44007, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165195}, {obj_id:44008, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184162}, {obj_id:44009, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159136}, {obj_id:44011, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190381}, {obj_id:44012, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162376}, {obj_id:44015, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145092}, {obj_id:44016, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177742}, {obj_id:44017, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147486}, {obj_id:44019, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177586}, {obj_id:44020, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156804}, {obj_id:44021, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196455}, {obj_id:44022, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165513}, {obj_id:44023, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179668}, {obj_id:44024, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157569}, {obj_id:44025, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220754}, {obj_id:44027, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194139}, {obj_id:44028, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164700}, {obj_id:44031, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181611}, {obj_id:44032, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158745}, {obj_id:44034, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237685}, {obj_id:44035, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195880}, {obj_id:44036, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233597}, {obj_id:44037, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188196}, {obj_id:44038, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161087}, {obj_id:44040, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177676}, {obj_id:44041, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155847}, {obj_id:44042, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181953}, {obj_id:44043, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157841}, {obj_id:44044, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184065}, {obj_id:44045, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159563}, {obj_id:44048, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224959}, {obj_id:44049, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192471}, {obj_id:44050, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163607}, {obj_id:44051, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239682}, {obj_id:44052, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198360}, {obj_id:44053, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167592}, {obj_id:44054, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194534}, {obj_id:44055, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164324}, {obj_id:44056, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194411}, {obj_id:44057, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163857}, {obj_id:44058, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179754}, {obj_id:44059, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157061}, {obj_id:44060, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184038}, {obj_id:44061, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159119}, {obj_id:44062, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163560}, {obj_id:44063, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152093}, {obj_id:44064, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194452}, {obj_id:44065, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164786}, {obj_id:44067, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198637}, {obj_id:44068, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167202}, {obj_id:44069, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186123}, {obj_id:44070, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160350}, {obj_id:44072, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192424}, {obj_id:44073, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162651}, {obj_id:44074, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192224}, {obj_id:44075, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164429}, {obj_id:44076, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171127}, {obj_id:44077, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192181}, {obj_id:44078, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163977}, {obj_id:44079, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186007}, {obj_id:44080, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160761}, {obj_id:44081, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181720}, {obj_id:44082, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158256}, {obj_id:44084, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186003}, {obj_id:44085, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160298}, {obj_id:44086, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229187}, {obj_id:44087, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231389}, {obj_id:44088, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188082}, {obj_id:44089, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161992}, {obj_id:44090, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136453}, {obj_id:44092, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182098}, {obj_id:44093, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157928}, {obj_id:44094, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188226}, {obj_id:44095, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160646}, {obj_id:44096, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196462}, {obj_id:44097, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165520}, {obj_id:44098, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184269}, {obj_id:44099, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158714}, {obj_id:44100, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:244056}, {obj_id:44102, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192270}, {obj_id:44103, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162638}, {obj_id:44104, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188186}, {obj_id:44105, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161593}, {obj_id:44106, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186031}, {obj_id:44107, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203
[2023-07-23 11:04:40.207858] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2032) [83051][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=1542] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:40.214933] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=19375, clean_start_pos=9000000, clean_num=200000)
[2023-07-23 11:04:40.215143] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081480043161, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.215233] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480159327 2023-07-23 03:04:40.159327|[process_begin] u=0 in_queue_time:7188578, receive_ts:1690081472970748, enqueue_ts:1690081480159310|[start_sql] u=0 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=33 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[pc_choose_plan] u=3 |[check_priv] u=4 |[plan_id] u=1 plan_id:45401|[do_open_plan_begin] u=1 plan_id:45401|[sql_start_stmt_begin] u=1 |[post_packet] u=9 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=55821 |[do_open_plan_end] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=13 |[process_end] u=2 run_ts:1690081480159329|total_timeu=55905)
[2023-07-23 11:04:40.223352] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2135) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=24] [PALF_DUMP](palf_id=1001, self="192.168.1.1:1001", [SlidingWindow]={palf_id:1001, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:406126649, max_lsn:704197632397, max_scn:{val:1690081466012354861}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:704197624828}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:704197620496}, last_submit_end_lsn:{lsn:704197624828}, last_submit_log_id:406126644, last_submit_log_pid:1, max_flushed_lsn:{lsn:704197620496}, max_flushed_end_lsn:{lsn:704197624828}, max_flushed_log_pid:1, committed_end_lsn:{lsn:704197620496}, last_slide_log_id:406126643, last_slide_scn:{val:1690081465406750436}, last_slide_lsn:{lsn:704197620374}, last_slide_end_lsn:{lsn:704197620496}, last_slide_log_pid:1, last_slide_log_accum_checksum:2486464045, last_fetch_end_lsn:{lsn:704239563536}, last_fetch_max_log_id:406128691, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd4653b0530}, [StateMgr]={this:0x7fd4653b33f0, self:"192.168.1.1:1001", palf_id:1001, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"0.0.0.0:0", leader_epoch:0, is_sync_enabled:true, allow_vote:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:406126644, is_changing_config_with_arb:false, reconfirm_start_time_us:-1, palf_role_change_cb:0x7fd4653b9670, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"192.168.1.1:1001", alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, prev_log_proposal_id:9223372036854775807, applied_alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, applied_alive_paxos_replica_num:3, applied_all_learnerlist:{learner_num:0, learner_array:[]}, prev_lsn:{lsn:18446744073709551615}, prev_mode_pid:9223372036854775807, state:0, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, region:"default_region", paxos_member_region_map:{}, register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7fd4653b15b0}, [ModeMgr]={palf_id:1001, self:"192.168.1.1:1001", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628709509541}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628709509541}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628709509541}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:1689906397955839346}, min_block_id:10151, base_lsn_for_block_gc:{lsn:703996121088}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:1, base_lsn:{lsn:703996121088}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1686557628709509541}}}, log_meta_storage:{log_tail:{lsn:1146880}, readable_log_tail:{lsn:1146880}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:704264540160}, min_scn:{val:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1001/meta", dir_fd:469, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65957888, block_header_serialize_buf_:0x7fd4653b5030}, log_storage:{log_tail:{lsn:704197624828}, readable_log_tail:{lsn:704197624828}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:704197435392}, min_scn:{val:1690081464462973371}, curr_block_id:10494, palf_id:1001}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1001/log", dir_fd:474, min_block_id:10151, max_block_id:10495, curr_writable_block_id:10494}, logical_block_size_:67104768, curr_block_writable_size_:66915332, block_header_serialize_buf_:0x7fd4653b6cb0}, palf_epoch:2, this:0x7fd4653b3af0}, [Reconfirm]={palf_id:1001, self:"192.168.1.1:1001", state:"INITED", new_proposal_id:9223372036854775807, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:0, majority_max_log_server:"0.0.0.0:0", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:18446744073709551615}, saved_end_lsn:{lsn:18446744073709551615}, last_submit_prepare_req_time_us:-1, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, has_notify_fetch:false, this:0x7fd4653b3530})
[2023-07-23 11:04:40.223513] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=159][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1001, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:40.223522] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2139) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=9] [PALF_DUMP](palf_id=1001, self="192.168.1.1:1001", ack_info_list=[])
[2023-07-23 11:04:40.223530] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1001, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:40.223539] INFO  [PALF] report_log_task_trace (log_sliding_window.cpp:3999) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=7] current log_task status(palf_id=1001, self="192.168.1.1:1001", log_id=406126644, log_task={header:{begin_lsn:{lsn:704197620496}, end_lsn:{lsn:18446744073709551615}, log_id:406126644, min_scn:{val:1690081465505288854}, max_scn:{val:1690081465505288854}, data_len:4276, proposal_id:1, prev_lsn:{lsn:704197620374}, prev_proposal_id:1, committed_end_lsn:{lsn:704197620496}, data_checksum:3720480937, accum_checksum:3552337702, is_padding_log:false, is_raw_write:false}, state_map:{val:26}, ref_cnt:0, gen_ts:1690081465556858, submit_wait_time:4, submit_ts:1690081465556862, flush_cost:692, flushed_ts:1690081465557554}, ack_info_list=[])
[2023-07-23 11:04:40.223558] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2135) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=18] [PALF_DUMP](palf_id=1003, self="192.168.1.1:1001", [SlidingWindow]={palf_id:1003, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:185506016, max_lsn:375365428400, max_scn:{val:1690081465143322589}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:375365428400}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:375365428288}, last_submit_end_lsn:{lsn:375365428400}, last_submit_log_id:185506016, last_submit_log_pid:54, max_flushed_lsn:{lsn:375365428288}, max_flushed_end_lsn:{lsn:375365428400}, max_flushed_log_pid:54, committed_end_lsn:{lsn:375365428166}, last_slide_log_id:185506014, last_slide_scn:{val:1690081465143322587}, last_slide_lsn:{lsn:375365426662}, last_slide_end_lsn:{lsn:375365428166}, last_slide_log_pid:54, last_slide_log_accum_checksum:2321785178, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd4653f2530}, [StateMgr]={this:0x7fd4653f53f0, self:"192.168.1.1:1001", palf_id:1003, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"PENDING", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:54}, leader:"0.0.0.0:0", leader_epoch:-1, is_sync_enabled:true, allow_vote:true, pending_end_lsn:{lsn:375365428400}, scan_disk_log_finished:true, last_check_start_id:185506015, is_changing_config_with_arb:false, reconfirm_start_time_us:1690075997426758, palf_role_change_cb:0x7fd4653fb670, allow_vote:true}, [ConfigMgr]={palf_id:1003, self:"192.168.1.1:1001", alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:2, proposal_id:54, prev:{config_version:{proposal_id:53, config_seq:54}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:54, config_seq:55}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:53, prev_lsn:{lsn:374655251805}, prev_mode_pid:54}, prev_log_proposal_id:53, applied_alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, applied_alive_paxos_replica_num:3, applied_all_learnerlist:{learner_num:0, learner_array:[]}, prev_lsn:{lsn:374655251805}, prev_mode_pid:54, state:0, persistent_config_version:{proposal_id:54, config_seq:55}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, region:"default_region", paxos_member_region_map:{}, register_time_us:1690075995458948, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1690075995458948, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7fd4653f35b0}, [ModeMgr]={palf_id:1003, self:"192.168.1.1:1001", applied_mode_meta:{version:1, proposal_id:54, mode_version:0, access_mode:1, ref_scn:{val:1686557628883962675}}, accepted_mode_meta:{version:1, proposal_id:54, mode_version:0, access_mode:1, ref_scn:{val:1686557628883962675}}, last_submit_mode_meta:{version:1, proposal_id:54, mode_version:0, access_mode:1, ref_scn:{val:1686557628883962675}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807}, [LogEngine]={palf_id:1003, is_inited:true, min_block_max_scn:{val:1689906400456444053}, min_block_id:5320, base_lsn_for_block_gc:{lsn:375249862656}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:54}, log_config_meta:{version:2, proposal_id:54, prev:{config_version:{proposal_id:53, config_seq:54}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:54, config_seq:55}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:53, prev_lsn:{lsn:374655251805}, prev_mode_pid:54}, log_snapshot_meta:{version:1, base_lsn:{lsn:375249862656}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:54, mode_version:0, access_mode:1, ref_scn:{val:1686557628883962675}}}, log_meta_storage:{log_tail:{lsn:1736704}, readable_log_tail:{lsn:1736704}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:375384072192}, min_scn:{val:0}, curr_block_id:0, palf_id:1003}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1003/meta", dir_fd:484, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65368064, block_header_serialize_buf_:0x7fd4653f7030}, log_storage:{log_tail:{lsn:375365428400}, readable_log_tail:{lsn:375365428400}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:375316967424}, min_scn:{val:1690081059652218233}, curr_block_id:5593, palf_id:1003}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1002/1003/log", dir_fd:485, min_block_id:5320, max_block_id:5594, curr_writable_block_id:5593}, logical_block_size_:67104768, curr_block_writable_size_:18643792, block_header_serialize_buf_:0x7fd4653f8cb0}, palf_epoch:4, this:0x7fd4653f5af0}, [Reconfirm]={palf_id:1003, self:"192.168.1.1:1001", state:"FINISHED", new_proposal_id:54, prepare_log_ack_list:1"10.1.129.154:1001";, curr_paxos_follower_list:2{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"192.168.1.1:1001", majority_max_accept_pid:53, majority_max_lsn:{lsn:374655252555}, saved_end_lsn:{lsn:374655252555}, last_submit_prepare_req_time_us:1690075997426853, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, has_notify_fetch:false, this:0x7fd4653f5530})
[2023-07-23 11:04:40.223691] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2139) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=131] [PALF_DUMP](palf_id=1003, self="192.168.1.1:1001", ack_info_list=[{member:{server:"10.1.129.154:1001", timestamp:1, flag:0}, last_ack_time_us:0, last_flushed_end_lsn:{lsn:375365428166}}, {member:{server:"10.1.129.155:1001", timestamp:1, flag:0}, last_ack_time_us:0, last_flushed_end_lsn:{lsn:375365428166}}, {member:{server:"192.168.1.1:1001", timestamp:1, flag:0}, last_ack_time_us:0, last_flushed_end_lsn:{lsn:375365428400}}])
[2023-07-23 11:04:40.223707] INFO  [PALF] report_log_task_trace (log_sliding_window.cpp:3999) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14] current log_task status(palf_id=1003, self="192.168.1.1:1001", log_id=185506015, log_task={header:{begin_lsn:{lsn:375365428166}, end_lsn:{lsn:375365428288}, log_id:185506015, min_scn:{val:1690081465143322588}, max_scn:{val:1690081465143322588}, data_len:66, proposal_id:54, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:375365428166}, data_checksum:940009244, accum_checksum:3763700303, is_padding_log:false, is_raw_write:false}, state_map:{val:30}, ref_cnt:0, gen_ts:1690081465324305, submit_wait_time:26, submit_ts:1690081465324331, flush_cost:166, flushed_ts:1690081465324497}, ack_info_list=[{member:{server:"10.1.129.154:1001", timestamp:1, flag:0}, last_ack_time_us:0, last_flushed_end_lsn:{lsn:375365428166}}, {member:{server:"10.1.129.155:1001", timestamp:1, flag:0}, last_ack_time_us:0, last_flushed_end_lsn:{lsn:375365428166}}, {member:{server:"192.168.1.1:1001", timestamp:1, flag:0}, last_ack_time_us:0, last_flushed_end_lsn:{lsn:375365428400}}])
[2023-07-23 11:04:40.230509] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.231900] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FB-0-0] [lt=8][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.231913] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FB-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.231921] WDIAG [SHARE.LOCATION] detect_ls_leaders_ (ob_ls_location_service.cpp:845) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FB-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", addr="10.1.129.154:1001", timeout=2000000)
[2023-07-23 11:04:40.231932] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FB-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.231935] WDIAG [SHARE.LOCATION] detect_ls_leaders_ (ob_ls_location_service.cpp:845) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FB-0-0] [lt=4][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", addr="10.1.129.155:1001", timeout=2000000)
[2023-07-23 11:04:40.232140] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FB-0-0] [lt=7][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=1193, arg={addr:"192.168.1.1:1001"})
[2023-07-23 11:04:40.232152] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3285][AutoLSLocRpc][T0][YB420A01819C-0005FDE71F3AE1FB-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=1193, arg={addr:"192.168.1.1:1001"})
[2023-07-23 11:04:40.233011] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=13] Cache replace map node details(ret=0, replace_node_count=0, replace_time=18265, replace_start_pos=22500000, replace_num=100000)
[2023-07-23 11:04:40.236827] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480145105, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908681}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480180889, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.236874] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=1] will sleep(sleep_us=93000, remain_us=20439664, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=93, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:40.237225] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B09-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.237241] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B09-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.240407] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196767}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480138778, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.240484] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=76] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196767}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480138778, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.240542] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=56] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480231484, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196767}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480138778, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.240633] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480240124 2023-07-23 03:04:40.240124|[process_begin] u=0 in_queue_time:6964884, receive_ts:1690081473275239, enqueue_ts:1690081480240105|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=4 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=100 |[check_priv] u=17 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=4 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=6 |[exec_plan_begin] u=0 |[exec_plan_end] u=4 |[do_open_plan_end] u=1 |[insert_open] u=3 |[insert_start] u=20 |[get_das_id_begin] u=35 |[get_das_id_end] u=0 |[insert_end] u=6 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=18 |[sync_auto_value] u=6 |[close_plan_begin] u=41 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=1 |[start_end_stmt] u=2 |[end_stmt] u=202 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=6 |[query_end] u=20 |[process_end] u=1 run_ts:1690081480240129|total_timeu=506)
[2023-07-23 11:04:40.241161] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F2-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.241187] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F2-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.241972] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F2-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.242738] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F2-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.243488] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F2-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.246475] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480246464]})
[2023-07-23 11:04:40.246493] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480246488]})
[2023-07-23 11:04:40.246496] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.246502] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.246510] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.246518] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.246634] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480246627]})
[2023-07-23 11:04:40.246657] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=5][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.246665] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.246673] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=4][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.246691] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=18][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.246704] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=13] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.246719] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=13] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.246724] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B05F-0-0] [lt=4] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480246723, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.246875] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F8-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.246891] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F8-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.247470] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C40-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.250247] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=22] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480250242})
[2023-07-23 11:04:40.250258] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC3B-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.250696] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=13][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.250713] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.250794] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480250183 2023-07-23 03:04:40.250183|[process_begin] u=0 in_queue_time:4296862, receive_ts:1690081475953320, enqueue_ts:1690081480250173|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=47 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[plc_serialize_begin] u=0 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=26 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=6 |[do_open_plan_end] u=1 |[get_das_id_begin] u=33 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=13 |[calc_partition_location_begin] u=341 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=5 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[remote_task_completed] u=39 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=18 |[close_das_task_begin] u=1 |[close_das_task_end] u=17 |[start_end_stmt] u=16 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=11 |[process_end] u=1 run_ts:1690081480250185|total_timeu=610)
[2023-07-23 11:04:40.251945] INFO  [OCCAM] ObOccamTimerTaskRAIIHandle (ob_occam_timer.h:363) [83036][T1002_Occam][T1002][Y0-0000000000000000-0-0] [lt=66] task handle constructed(*this={this:0x7fd4b094b9e0, task:NULL, is_inited:false, is_running:false})
[2023-07-23 11:04:40.252511] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:644) [3308][Occam][T1][YB420A01819C-0005FDE625B7976B-0-0] [lt=0] inner session no need register to deadlock(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4122, exec_ctx.get_errcode()="OB_RPC_POST_ERROR", session={this:0x7fd4cac180d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fe41f0}, desc=NULL, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.252534] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [3308][Occam][T1][YB420A01819C-0005FDE625B7976B-0-0] [lt=23] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4122, exec_ctx.get_errcode()="OB_RPC_POST_ERROR", session={this:0x7fd4cac180d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fe41f0}, desc=NULL, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.252551] INFO  [STORAGE.TRANS] rollback_to_global_implicit_savepoint_ (ob_tx_api.cpp:1023) [3308][Occam][T1][YB420A01819C-0005FDE625B7976B-0-0] [lt=16] add tx part with extra_touched_ls(extra_touched_ls=[{id:1}], tx.tx_id={txid:44908682})
[2023-07-23 11:04:40.252568] INFO  [STORAGE.TRANS] abort_tx_ (ob_trans_service_v4.cpp:575) [3308][Occam][T1][YB420A01819C-0005FDE625B7976B-0-0] [lt=0] abort tx(ret=0, *this={is_inited_:true, tenant_id_:1, this:0x7fd550e04030}, tx={this:0x7fd4f4fe41f0, tx_id:{txid:44908682}, state:6, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081480251515, op_sn:3, alloc_ts:1690081480251513, active_ts:1690081480251513, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1690081510251507, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"0.0.0.0:0", epoch:-1, first_scn:9223372036854775807, last_scn:9223372036854775807, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:4, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, cause=4)
[2023-07-23 11:04:40.252598] INFO  [STORAGE.TRANS] start_epoch_ (ob_tx_api.cpp:1736) [3308][Occam][T1][YB420A01819C-0005FDE625B7976B-0-0] [lt=29] tx start new epoch(ret=0, tx={this:0x7fd4f4fe41f0, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:0, op_sn:4, alloc_ts:1690081480251513, active_ts:0, commit_ts:0, finish_ts:0, timeout_us:0, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:40.252620] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [3308][Occam][T1][YB420A01819C-0005FDE625B7976B-0-0] [lt=19] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480251515, tx_desc={this:0x7fd4f4fe41f0, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:0, op_sn:4, alloc_ts:1690081480251513, active_ts:0, commit_ts:0, finish_ts:0, timeout_us:0, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[{id:1}], cflict_txs:[]}, rollback=true, session={this:0x7fd4cac180d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fe41f0}, exec_ctx.get_errcode()=-4122)
[2023-07-23 11:04:40.252658] INFO  [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1915) [3308][Occam][T0][Y0-0000000000000000-0-0] [lt=0] execute write sql(ret=-4122, tenant_id=1, affected_rows=0, sql=INSERT INTO __all_server_event_history (gmt_create, module, event, name1, value1, name2, value2, name3, value3, name4, value4, svr_ip, svr_port) VALUES (usec_to_time(1690081480251988), 'ELECTION', 'vote', 'TENANT_ID', 1002, 'LS_ID', 1001, 'INFO', 'vote for "192.168.1.1:1001", reason:the only request', 'HAPPENED_TIME', '2023-07-23 11:04:40.251940', '192.168.1.1', 2882))
[2023-07-23 11:04:40.257192] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.257203] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.257230] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14795596, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.257240] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.259868] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1951-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.272164] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.272249] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.280874] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.280974] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480280453 2023-07-23 03:04:40.280453|[process_begin] u=0 in_queue_time:4256627, receive_ts:1690081476023826, enqueue_ts:1690081480280445|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=3 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=25 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=2 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=14 |[calc_partition_location_begin] u=335 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=5 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=25 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=4 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[close_plan_begin] u=53 |[close_das_task_begin] u=0 |[close_das_task_end] u=20 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=7 |[process_end] u=1 run_ts:1690081480280457|total_timeu=521)
[2023-07-23 11:04:40.281013] INFO  process_request (ob_th_worker.cpp:296) [45604][T1_L0_G100][T1][Y0-0000000000000000-0-0] [lt=36] thd_flag=1 total=2079744 used=1488 pm_hold=6291456
[2023-07-23 11:04:40.290785] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196768}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480189321, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.290839] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=53] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196768}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480189321, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.290879] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=38] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480251517, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196768}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480189321, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.290941] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480290573 2023-07-23 03:04:40.290573|[process_begin] u=0 in_queue_time:7630437, receive_ts:1690081472660135, enqueue_ts:1690081480290560|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=75 |[check_priv] u=12 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=3 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=4 |[exec_plan_begin] u=1 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=3 |[value_start_calc_row] u=4 |[value_after_calc_row] u=19 |[insert_start] u=0 |[get_das_id_begin] u=19 |[get_das_id_end] u=0 |[insert_end] u=5 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=13 |[sync_auto_value] u=4 |[close_plan_begin] u=31 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=139 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=15 |[process_end] u=1 run_ts:1690081480290576|total_timeu=366)
[2023-07-23 11:04:40.291765] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879362-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.292719] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.292742] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=14777354, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.292736] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.292758] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.292763] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=14886039, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.292760] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879362-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.292774] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1001}, delta=14885801, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.292774] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.292784] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.292795] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1002}, delta=15215088, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.292805] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.292816] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1003}, delta=15149229, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.292822] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=5] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.293742] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879362-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.294714] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879362-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.300705] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.300740] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480300636 2023-07-23 03:04:40.300636|[process_begin] u=0 in_queue_time:5075945, receive_ts:1690081475224690, enqueue_ts:1690081480300626|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=22 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=1 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[parse_job_begin] u=1 |[parse_job_end] u=4 |[remote_task_completed] u=41 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29255}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=10 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=1 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=7 |[process_end] u=0 run_ts:1690081480300638|total_timeu=103)
[2023-07-23 11:04:40.300818] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [83391][T1001_HAService][T1001][Y0-0000000000000000-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1}])
[2023-07-23 11:04:40.315277] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3385][sql_nio7][T0][Y0-0000000000000000-0-0] [lt=9] new mysql sessid created(conn.sessid_=3222112684, support_ssl=false)
[2023-07-23 11:04:40.315318] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3385][sql_nio7][T0][Y0-0000000000000000-0-0] [lt=22] sm conn init succ(conn.sessid_=3222112684, sess.client_addr_="192.168.1.1:43074")
[2023-07-23 11:04:40.315330] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3385][sql_nio7][T0][Y0-0000000000000000-0-0] [lt=9] accept one succ(*s={this:0x7fd069dba030, fd:361, err:0, last_decode_time_:0, last_write_time_:1690081480315318, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.315426] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3385][sql_nio7][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd069dba030, fd:361, err:5, last_decode_time_:0, last_write_time_:1690081480315318, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.315442] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3385][sql_nio7][T0][Y0-0000000000000000-0-0] [lt=14] kill and revert session(conn.sessid_=3222112684, proxy_sessid=0, server_id=3, ret=0)
[2023-07-23 11:04:40.315450] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:795) [3385][sql_nio7][T0][Y0-0000000000000000-0-0] [lt=8] sock ref clean, do destroy(*s={this:0x7fd069dba030, fd:361, err:5, last_decode_time_:0, last_write_time_:1690081480315318, read_buffer_.get_consume_sz():0, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.315460] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3385][sql_nio7][T0][Y0-0000000000000000-0-0] [lt=8] connection close(sessid=3222112684, proxy_sessid=0, tenant_id=0, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:40.320887] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.328236] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480251516, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:132, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.328276] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] will sleep(sleep_us=100000, remain_us=14841571, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=129, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:40.329470] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:116) [83143][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=10] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000)
[2023-07-23 11:04:40.329491] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [83143][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=17] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0)
[2023-07-23 11:04:40.329506] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [83143][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=13] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0)
[2023-07-23 11:04:40.329515] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [83143][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=9] [tabletchange] task check ls(ls->get_ls_id()={id:1002}, tablet_persist_trigger=0)
[2023-07-23 11:04:40.329526] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:150) [83143][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1003}, tablet_persist_trigger=0)
[2023-07-23 11:04:40.329534] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:214) [83143][T1002_ObTimer][T1002][Y0-0000000000000000-0-0] [lt=6] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=4, times=9385)
[2023-07-23 11:04:40.330022] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.330562] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.346972] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480346962]})
[2023-07-23 11:04:40.346990] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480346985]})
[2023-07-23 11:04:40.347007] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=12][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.347025] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.347038] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=12][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.347059] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.347069] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.347133] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480347126]})
[2023-07-23 11:04:40.347228] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=16][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.347247] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=17][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.347268] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=14][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.347316] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=48][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.347349] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=31] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.347390] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=36] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.347405] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B060-0-0] [lt=11] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480347402, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.347645] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1F9-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.347834] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C24-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.351179] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=26] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480351166})
[2023-07-23 11:04:40.351182] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC3C-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.351311] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196769}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480239916, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.351362] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=51] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196769}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480239916, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.351402] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=39] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480346215, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196769}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480239916, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.351461] TRACE [TRACE] after_process (obmp_base.cpp:156) [80220][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480351107 2023-07-23 03:04:40.351107|[process_begin] u=0 in_queue_time:7075867, receive_ts:1690081473275239, enqueue_ts:1690081480351096|[start_sql] u=1 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=70 |[check_priv] u=11 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=3 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=4 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=5 |[value_after_calc_row] u=9 |[insert_start] u=0 |[get_das_id_begin] u=17 |[get_das_id_end] u=0 |[insert_end] u=4 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=11 |[sync_auto_value] u=4 |[close_plan_begin] u=45 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=134 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=14 |[process_end] u=1 run_ts:1690081480351110|total_timeu=352)
[2023-07-23 11:04:40.351641] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079326-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.351752] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F7-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.351768] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F7-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.351996] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.352007] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079326-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.352064] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F3-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.352144] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480351076 2023-07-23 03:04:40.351076|[process_begin] u=0 in_queue_time:4397755, receive_ts:1690081475953320, enqueue_ts:1690081480351066|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=67 |[plc_serialize_begin] u=3 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=1 |[plc_serialize_begin] u=0 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=39 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=3 |[pc_choose_plan] u=2 |[check_priv] u=4 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=16 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=8 |[do_open_plan_end] u=1 |[get_das_id_begin] u=77 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=20 |[calc_partition_location_begin] u=609 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=9 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=47 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=2 |[close_plan_begin] u=6 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=6 |[close_plan_begin] u=32 |[close_das_task_begin] u=1 |[close_das_task_end] u=30 |[start_end_stmt] u=26 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=20 |[process_end] u=1 run_ts:1690081480351080|total_timeu=1067)
[2023-07-23 11:04:40.355777] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3324][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/1, request done=303730708/342333753, request doing=0/0)
[2023-07-23 11:04:40.357248] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.357277] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14895566, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.357288] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.357855] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1FA-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.360615] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1952-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.373017] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83283][T1001_LogUpdate][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.376790] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=1] start stmt(ret=-4283, auto_commit=true, session_id=3222118872, snapshot={this:0x7fd039ace628, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd438065c60, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222118872, assoc_session_id:3222118872, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:-1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081480214612, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081472970748, use_das=false, nested_level=0, session={this:0x7fd4f8c557f8, id:3222118872, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_psg_data_center_test", user:"uathsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd438065c60}, plan=0x7fd1dd1ae050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.376853] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480320838 2023-07-23 03:04:40.320838|[process_begin] u=0 in_queue_time:7350090, receive_ts:1690081472970748, enqueue_ts:1690081480320829|[start_sql] u=0 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222118872, tenant_id:1002|[calc_partition_location_begin] u=25 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[pc_choose_plan] u=3 |[check_priv] u=5 |[plan_id] u=0 plan_id:45401|[do_open_plan_begin] u=1 plan_id:45401|[sql_start_stmt_begin] u=1 |[post_packet] u=7 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=55950 |[do_open_plan_end] u=0 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=9 |[process_end] u=1 run_ts:1690081480320840|total_timeu=56014)
[2023-07-23 11:04:40.382034] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.382162] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480381345 2023-07-23 03:04:40.381345|[process_begin] u=0 in_queue_time:4357518, receive_ts:1690081476023826, enqueue_ts:1690081480381335|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=5 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=38 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=0 |[check_priv] u=4 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_das_id_begin] u=3 |[get_das_id_end] u=1 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=20 |[calc_partition_location_begin] u=560 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=9 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=38 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=5 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=57 |[close_das_task_begin] u=1 |[close_das_task_end] u=29 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=10 |[process_end] u=1 run_ts:1690081480381351|total_timeu=816)
[2023-07-23 11:04:40.383350] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3388][sql_nio10][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd3e064f290, fd:470, err:5, last_decode_time_:1690081472970748, last_write_time_:1690080852194978, read_buffer_.get_consume_sz():2115, get_pending_flag():1, get_trace_id():YB420A01819C-0005FDE730AB57FA-0-0})
[2023-07-23 11:04:40.383371] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3388][sql_nio10][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3222118872, proxy_sessid=720999913969918232, server_id=3, ret=0)
[2023-07-23 11:04:40.383951] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd3341e1290, fd:389, err:5, last_decode_time_:1690081464683798, last_write_time_:1690081464683964, read_buffer_.get_consume_sz():5929775, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.383973] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=17] kill and revert session(conn.sessid_=3222114616, proxy_sessid=720999922559852502, server_id=3, ret=0)
[2023-07-23 11:04:40.383983] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:795) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] sock ref clean, do destroy(*s={this:0x7fd3341e1290, fd:389, err:5, last_decode_time_:1690081464683798, last_write_time_:1690081464683964, read_buffer_.get_consume_sz():5929775, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.383999] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10] unlock session of tenant(conn.sessid_=3222114616, proxy_sessid=720999922559852502, conn.tenant_id_=1002)
[2023-07-23 11:04:40.384007] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=8] connection close(sessid=3222114616, proxy_sessid=720999922559852502, tenant_id=1002, server_id=3, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:40.384029] INFO  [SERVER] run (obmp_disconnect.cpp:78) [83260][T1002_L0_G0][T1002][Y0-0000000000000000-0-0] [lt=12] free session successfully(sessid=3222114616)
[2023-07-23 11:04:40.384313] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd4f5acad70, fd:382, err:5, last_decode_time_:1690080927720139, last_write_time_:1690080927720620, read_buffer_.get_consume_sz():408, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.384332] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=15] kill and revert session(conn.sessid_=3222108299, proxy_sessid=720999922559852502, server_id=3, ret=0)
[2023-07-23 11:04:40.384338] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:795) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=6] sock ref clean, do destroy(*s={this:0x7fd4f5acad70, fd:382, err:5, last_decode_time_:1690080927720139, last_write_time_:1690080927720620, read_buffer_.get_consume_sz():408, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.384351] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=7] unlock session of tenant(conn.sessid_=3222108299, proxy_sessid=720999922559852502, conn.tenant_id_=1002)
[2023-07-23 11:04:40.384357] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=6] connection close(sessid=3222108299, proxy_sessid=720999922559852502, tenant_id=1002, server_id=3, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:40.384375] INFO  [SERVER] run (obmp_disconnect.cpp:78) [83222][T1002_L0_G0][T1002][Y0-0000000000000000-0-0] [lt=9] free session successfully(sessid=3222108299)
[2023-07-23 11:04:40.385896] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480251519, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908683}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480329392, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.385944] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=1] will sleep(sleep_us=94000, remain_us=20290594, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=94, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:40.392747] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1}, delta=14877331, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.392767] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.392788] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.392787] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1001}, delta=14985778, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.392797] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.392804] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.392811] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1002}, delta=15315065, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.392819] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.392831] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14986016, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.392835] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] get wrs ts(ls_id={id:1003}, delta=15249206, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.392844] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.392842] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.392848] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.392857] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=9][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.392876] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.392883] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.393042] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=11][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.393055] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=11][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.393070] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=9][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.393096] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=26][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.393120] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=23] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.393147] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=25] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.393158] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B061-0-0] [lt=8] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480393156, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.394348] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=7] new mysql sessid created(conn.sessid_=3222111486, support_ssl=false)
[2023-07-23 11:04:40.394377] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=10] sm conn init succ(conn.sessid_=3222111486, sess.client_addr_="192.168.1.1:43076")
[2023-07-23 11:04:40.394388] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3393][sql_nio15][T0][Y0-0000000000000000-0-0] [lt=7] accept one succ(*s={this:0x7fd45154c030, fd:368, err:0, last_decode_time_:0, last_write_time_:1690081480394376, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.394989] INFO  [SERVER] process (obmp_connect.cpp:369) [3197][MysqlQueueTh5][T1002][Y0-0005FDE6FBAC52DF-0-0] [lt=15] MySQL LOGIN(direct_client_ip="192.168.1.1", client_ip=10.1.128.25, tenant_name=ob01, tenant_id=1002, user_name=uathsd, host_name=%, sessid=3222111486, proxy_sessid=720999922559852561, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=10461711, proxy_capability=129887, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:40.395324] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3323][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=1/1, request done=303730835/342934136, request doing=0/0)
[2023-07-23 11:04:40.395545] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3329][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/1, request done=303730959/343338626, request doing=0/0)
[2023-07-23 11:04:40.395951] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=7] new mysql sessid created(conn.sessid_=3222109704, support_ssl=false)
[2023-07-23 11:04:40.395980] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=12] sm conn init succ(conn.sessid_=3222109704, sess.client_addr_="192.168.1.1:43078")
[2023-07-23 11:04:40.395990] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=8] accept one succ(*s={this:0x7fd422772b60, fd:374, err:0, last_decode_time_:0, last_write_time_:1690081480395979, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.397001] INFO  [SERVER] process (obmp_connect.cpp:369) [3195][MysqlQueueTh3][T1002][Y0-0005FDE6FBEC4EB2-0-0] [lt=36] MySQL LOGIN(direct_client_ip="192.168.1.1", client_ip=10.1.128.25, tenant_name=ob01, tenant_id=1002, user_name=uathsd, host_name=%, sessid=3222109704, proxy_sessid=720999922559852561, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=10461711, proxy_capability=129887, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:40.397117] INFO  [SERVER] process (obmp_init_db.cpp:53) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB5804-0-0] [lt=10] init db(db_name=ob_db_psg_data_center_test)
[2023-07-23 11:04:40.397156] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB5804-0-0] [lt=12] success add system var to ok pack(str_kv={key:"character_set_database", value:"45"}, change_var={id_:5, old_val_:{"BIGINT":45}}, new_val={"BIGINT":45}, session.get_sessid()=3222109704, session.get_proxy_sessid()=720999922559852561, change_var.id_=5)
[2023-07-23 11:04:40.397174] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB5804-0-0] [lt=17] success add system var to ok pack(str_kv={key:"collation_database", value:"45"}, change_var={id_:10, old_val_:{"BIGINT":45}}, new_val={"BIGINT":45}, session.get_sessid()=3222109704, session.get_proxy_sessid()=720999922559852561, change_var.id_=10)
[2023-07-23 11:04:40.397422] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB5805-0-0] [lt=11] success add system var to ok pack(str_kv={key:"ob_enable_transmission_checksum", value:"1"}, change_var={id_:10035, old_val_:{"BIGINT":1}}, new_val={"BIGINT":1}, session.get_sessid()=3222109704, session.get_proxy_sessid()=720999922559852561, change_var.id_=10035)
[2023-07-23 11:04:40.398481] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB5807-0-0] [lt=15] success add system var to ok pack(str_kv={key:"character_set_results", value:""}, change_var={id_:6, old_val_:{"BIGINT":45}}, new_val={"NULL":"NULL"}, session.get_sessid()=3222109704, session.get_proxy_sessid()=720999922559852561, change_var.id_=6)
[2023-07-23 11:04:40.399902] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB5809-0-0] [lt=16] success add system var to ok pack(str_kv={key:"sql_mode", value:"14680064"}, change_var={id_:15, old_val_:{"BIGINT UNSIGNED":12582912}}, new_val={"BIGINT UNSIGNED":14680064}, session.get_sessid()=3222109704, session.get_proxy_sessid()=720999922559852561, change_var.id_=15)
[2023-07-23 11:04:40.401560] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=56][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480393156})
[2023-07-23 11:04:40.401584] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=25][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480393156}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.401604] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=20][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.401613] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.401619] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480393156}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:40.401647] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=28][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:40.401660] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:40.401669] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:40.401675] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:40.401696] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196770}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480290443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.401730] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=34] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196770}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480290443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.401758] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480392530, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196770}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480290443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.401799] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480401553 2023-07-23 03:04:40.401553|[process_begin] u=0 in_queue_time:7741417, receive_ts:1690081472660135, enqueue_ts:1690081480401543|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=55 |[check_priv] u=8 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[insert_start] u=14 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=17 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=90 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=10 |[process_end] u=1 run_ts:1690081480401555|total_timeu=245)
[2023-07-23 11:04:40.402590] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879363-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.403457] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:40.403510] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=22][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480393156})
[2023-07-23 11:04:40.403531] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=21][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480393156}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.403538] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879363-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.403550] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=19][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.403559] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.403564] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.403572] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:40.403582] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:40.403588] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.403596] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.403600] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.403607] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:40.403611] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:40.403618] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:40.403622] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:40.403631] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:40.403638] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:40.403645] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:40.403652] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:40.403665] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:52, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:40.403676] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.403684] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=4821006)
[2023-07-23 11:04:40.403711] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949275415, cache_obj->added_lc()=false, cache_obj->get_object_id()=20502, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.403731] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9] [packet retry query](TRACE=begin_ts=1690081480401523 2023-07-23 03:04:40.401523|[process_begin] u=0 in_queue_time:5176833, receive_ts:1690081475224690, enqueue_ts:1690081480401513|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=28 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=117 |[transform_with_outline_begin] u=69 |[transform_with_outline_end] u=1 |[resolve_begin] u=2 |[resolve_end] u=582 |[transform_begin] u=23 |[optimize_begin] u=922 |[tl_calc_part_id_end] u=171 |[get_location_cache_begin] u=1 |[get_location_cache_begin] u=219 |[query_end] u=65 |[process_end] u=1 run_ts:1690081480401525|total_timeu=2207)
[2023-07-23 11:04:40.404429] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879363-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.405326] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879363-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.414098] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3327][RpcIO][T0][YB420A01819C-0005FDE62EAFF135-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=1/1, request done=303731152/342559127, request doing=0/0)
[2023-07-23 11:04:40.415178] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73136676044, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:40.416351] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Wash time detail, (compute_wash_size_time=83, refresh_score_time=679, wash_time=478)
[2023-07-23 11:04:40.424554] INFO  [STORAGE.BLKMGR] inspect_bad_block (ob_block_manager.cpp:1460) [3293][BlkMgr][T0][Y0-0000000000000000-0-0] [lt=16] check macro block(block_info={ref_cnt:1, is_free:false, access_time:1689184904407215}, time_interval=896576017334)
[2023-07-23 11:04:40.428400] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.430618] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.435666] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=9] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=19304, clean_start_pos=9200000, clean_num=200000)
[2023-07-23 11:04:40.439246] INFO  [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:221) [3271][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=14] [ASYNC TASK QUEUE](queue_size=0)
[2023-07-23 11:04:40.442481] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=9389, replace_start_pos=22600000, replace_num=100000)
[2023-07-23 11:04:40.447484] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480447471]})
[2023-07-23 11:04:40.447499] INFO  [STORAGE.TRANS] operator() (ob_ts_mgr.h:171) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts functor(ret=-4122, ret="OB_RPC_POST_ERROR", gts_tenant_info={v:1})
[2023-07-23 11:04:40.447509] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.447518] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480447509]})
[2023-07-23 11:04:40.447519] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.447531] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.447537] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.447673] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.447687] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=12][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.447699] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480447690]})
[2023-07-23 11:04:40.447702] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=8][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.447724] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=22][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.447743] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=17] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.447764] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=18] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.447774] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B062-0-0] [lt=6] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480447772, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.447921] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1FB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.448156] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF4-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.448171] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF4-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.452060] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=22] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480452049})
[2023-07-23 11:04:40.452065] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC3D-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.453076] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.453251] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480451946 2023-07-23 03:04:40.451946|[process_begin] u=0 in_queue_time:4498625, receive_ts:1690081475953320, enqueue_ts:1690081480451933|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=78 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=6 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=45 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=3 |[pc_choose_plan] u=2 |[check_priv] u=5 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=3 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=1 |[exec_plan_end] u=9 |[do_open_plan_end] u=1 |[get_das_id_begin] u=83 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=24 |[calc_partition_location_begin] u=797 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=12 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[remote_task_completed] u=51 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=7 |[start_end_stmt] u=1 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[close_plan_begin] u=38 |[close_das_task_begin] u=1 |[close_das_task_end] u=38 |[start_end_stmt] u=32 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=23 |[process_end] u=1 run_ts:1690081480451950|total_timeu=1303)
[2023-07-23 11:04:40.455817] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:351) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ======
[2023-07-23 11:04:40.455830] INFO  [PALF] get_disk_usage (palf_env_impl.cpp:781) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=11] get_disk_usage(ret=0, capacity(MB):=16384, used(MB):=13093)
[2023-07-23 11:04:40.456643] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x7fd527c04030, ls_id:{id:1}, tenant_id:1, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1690048801845283338}, last_push_gc_task_ts_:1690048805013194, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.456661] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=18] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd525424080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1690048801913851}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd525424080, snapshot_version:{val:1690048801913851}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.456684] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=18] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:0}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:0}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1})
[2023-07-23 11:04:40.456697] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7fd52c1de1b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7fd52c1e3290, table_type:1, memtable_head:47, memtable_tail:48, t3m:0x7fd5aebe2030, tables:[null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 0x7fd525414b00]}, is_freezing:false, ls_id:{id:1}, tx_data_table:0x7fd52c1e4650, ls_tablet_svr:0x7fd52c1de190, slice_allocator:0x7fd52c1e4690}, min_rec_scn={val:1690048801911385819}, ls_id_={id:1})
[2023-07-23 11:04:40.457306] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.457329] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14995558, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.457339] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.457386] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=20] get rec log scn(service_type_=0, rec_log_ts={val:1690081457831266372})
[2023-07-23 11:04:40.457394] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903})
[2023-07-23 11:04:40.457399] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=2, rec_log_ts={val:4611686018427387903})
[2023-07-23 11:04:40.457407] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690048801911385819}, checkpoint_scn_in_ls_meta={val:1690048801911385819}, ls_id={id:1}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:40.457418] INFO  [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:264) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=8] cannot_recycle_log_size statistics(cannot_recycle_log_size=240296131, threshold=5153960755)
[2023-07-23 11:04:40.457431] INFO  [STORAGE] flush_if_need_ (ob_ls.cpp:1564) [3731][T1_CKClogDisk][T1][Y0-0000000000000000-0-0] [lt=6] the ls no need flush to advance_checkpoint(get_ls_id()={id:1}, need_flush=false)
[2023-07-23 11:04:40.460066] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83150][T1002_ArcSrv][T1001][YB420A01819C-0005FDE628578DB5-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.460080] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83150][T1002_ArcSrv][T1001][YB420A01819C-0005FDE628578DB5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.460467] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83150][T1002_ArcSrv][T1001][YB420A01819C-0005FDE628578DB5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.461187] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1953-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.461488] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3222122363, support_ssl=false)
[2023-07-23 11:04:40.461514] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10] sm conn init succ(conn.sessid_=3222122363, sess.client_addr_="192.168.1.1:43080")
[2023-07-23 11:04:40.461524] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3379][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=8] accept one succ(*s={this:0x7fd4833f8390, fd:382, err:0, last_decode_time_:0, last_write_time_:1690081480461513, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.462228] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.462332] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196771}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480350429, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.462388] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=55] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196771}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480350429, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.462434] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=44] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480447233, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196771}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480350429, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.462507] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480462063 2023-07-23 03:04:40.462063|[process_begin] u=0 in_queue_time:7186823, receive_ts:1690081473275239, enqueue_ts:1690081480462050|[start_sql] u=1 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=81 |[check_priv] u=13 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=4 plan_id:28026|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=4 |[exec_plan_begin] u=1 |[exec_plan_end] u=4 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=5 |[value_after_calc_row] u=11 |[insert_start] u=0 |[get_das_id_begin] u=20 |[get_das_id_end] u=0 |[insert_end] u=5 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=33 |[sync_auto_value] u=5 |[close_plan_begin] u=63 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=155 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=5 |[query_end] u=17 |[process_end] u=1 run_ts:1690081480462067|total_timeu=442)
[2023-07-23 11:04:40.462573] INFO  [SERVER] process (obmp_connect.cpp:369) [3192][MysqlQueueTh0][T1002][Y0-0005FDE6FD2C2EC8-0-0] [lt=34] MySQL LOGIN(direct_client_ip="192.168.1.1", client_ip=10.1.128.25, tenant_name=ob01, tenant_id=1002, user_name=clhsd, host_name=%, sessid=3222122363, proxy_sessid=720999922559852562, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=10461711, proxy_capability=129887, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:40.462955] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F8-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.463136] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F4-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.463957] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3388][sql_nio10][T0][Y0-0000000000000000-0-0] [lt=8] new mysql sessid created(conn.sessid_=3222112341, support_ssl=false)
[2023-07-23 11:04:40.463994] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3388][sql_nio10][T0][Y0-0000000000000000-0-0] [lt=16] sm conn init succ(conn.sessid_=3222112341, sess.client_addr_="192.168.1.1:43082")
[2023-07-23 11:04:40.464001] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F4-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.464008] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3388][sql_nio10][T0][Y0-0000000000000000-0-0] [lt=10] accept one succ(*s={this:0x7fd3fb610150, fd:389, err:0, last_decode_time_:0, last_write_time_:1690081480463994, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.464826] INFO  [SERVER] process (obmp_connect.cpp:369) [3193][MysqlQueueTh1][T1002][Y0-0005FDE6FA9C6EDB-0-0] [lt=20] MySQL LOGIN(direct_client_ip="192.168.1.1", client_ip=10.1.128.25, tenant_name=ob01, tenant_id=1002, user_name=clhsd, host_name=%, sessid=3222112341, proxy_sessid=720999922559852562, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=10461711, proxy_capability=129887, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:40.464837] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F4-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.464976] INFO  [SERVER] process (obmp_init_db.cpp:53) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB580A-0-0] [lt=16] init db(db_name=ob_db_mysql_hsd)
[2023-07-23 11:04:40.465013] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB580A-0-0] [lt=13] success add system var to ok pack(str_kv={key:"character_set_database", value:"45"}, change_var={id_:5, old_val_:{"BIGINT":45}}, new_val={"BIGINT":45}, session.get_sessid()=3222112341, session.get_proxy_sessid()=720999922559852562, change_var.id_=5)
[2023-07-23 11:04:40.465034] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB580A-0-0] [lt=20] success add system var to ok pack(str_kv={key:"collation_database", value:"45"}, change_var={id_:10, old_val_:{"BIGINT":45}}, new_val={"BIGINT":45}, session.get_sessid()=3222112341, session.get_proxy_sessid()=720999922559852562, change_var.id_=10)
[2023-07-23 11:04:40.465312] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB580B-0-0] [lt=18] success add system var to ok pack(str_kv={key:"ob_enable_transmission_checksum", value:"1"}, change_var={id_:10035, old_val_:{"BIGINT":1}}, new_val={"BIGINT":1}, session.get_sessid()=3222112341, session.get_proxy_sessid()=720999922559852562, change_var.id_=10035)
[2023-07-23 11:04:40.466704] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB580D-0-0] [lt=18] success add system var to ok pack(str_kv={key:"character_set_results", value:""}, change_var={id_:6, old_val_:{"BIGINT":45}}, new_val={"NULL":"NULL"}, session.get_sessid()=3222112341, session.get_proxy_sessid()=720999922559852562, change_var.id_=6)
[2023-07-23 11:04:40.468490] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [83222][T1002_L0_G0][T1002][YB420A01819C-0005FDE730AB580F-0-0] [lt=20] success add system var to ok pack(str_kv={key:"sql_mode", value:"14680064"}, change_var={id_:15, old_val_:{"BIGINT UNSIGNED":12582912}}, new_val={"BIGINT UNSIGNED":14680064}, session.get_sessid()=3222112341, session.get_proxy_sessid()=720999922559852562, change_var.id_=15)
[2023-07-23 11:04:40.471070] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3694][T1_FreInfoReloa][T1][YB420A01819C-0005FDE72A1D929D-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.471085] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3694][T1_FreInfoReloa][T1][YB420A01819C-0005FDE72A1D929D-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.471557] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3694][T1_FreInfoReloa][T1][YB420A01819C-0005FDE72A1D929E-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.474391] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3633][T1_LogUpdater][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.474467] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.474484] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=15][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.474507] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.474522] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=14][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.474679] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=16][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.474698] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=18][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.474720] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=14][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.474758] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=38][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.474794] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=34] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.474837] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=39] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.474856] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B063-0-0] [lt=15] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480474853, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.480066] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.482252] WDIAG [SERVER] process (obmp_query.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=43][errcode=-5066] session has been killed(session.get_session_state()=4, sql=select
     
    tid, update_time, `status`, pti_flight_tid,pti_market_flight_tid, pti_international, pti_is_group, pti_ticket_no,
    pti_ticket_type, pti_iss_time, pti_iss_office, pti_agent_code, pti_ticket_code, pti_ticket_status, 
    pti_exchange_tkt_no, pti_exchange_time, pti_pnr, pti_pnr_clear_time, pti_psg_type, 
    pti_psg_tid, pti_is_protect, pti_sale_cabin, pti_cabin, pti_segment_status, pti_cabin_action, 
    pti_phone, pti_other_phone, pti_tel, pti_mobile, pti_fare_price, pti_total_price, 
    pti_sale_channel, pti_acc_date, pti_acc_agent, pti_check_in_mode, pti_boarding_no,
    pti_seat_no, pti_is_risk, pti_is_vip, pti_luggage_allowance, pti_fare_basis, pti_tc, pti_cn_tax, pti_yq_tax
   
    from passenger_tkt_info
    where 1=1 
     
     
     
     
      and pti_flight_tid = 4170080617828875
     
     
     
     
     
      and pti_ticket_no = '7819106997474', session.get_sessid()=3222118872, proxy_sessid=720999913969918232, ret=-5066)
[2023-07-23 11:04:40.482314] INFO  [SERVER] send_error_packet (obmp_packet_sender.cpp:311) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=20] sending error packet(err=-5066, extra_err_info=NULL, lbt()="0xdaaa2a5 0x7620021 0x75e383a 0x3a1355b 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.482342] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=14] success add system var to ok pack(str_kv={key:"ob_statement_trace_id", value:"YB420A01819C-0005FDE730AB57FA-0-0"}, change_var={id_:10037, old_val_:{"VARCHAR":"Y0-0", collation:"utf8mb4_general_ci", coercibility:"SYSCONST"}}, new_val={"VARCHAR":"YB420A01819C-0005FDE730AB57FA-0-0", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, session.get_sessid()=3222118872, session.get_proxy_sessid()=720999913969918232, change_var.id_=10037)
[2023-07-23 11:04:40.482393] INFO  [SERVER] send_error_packet (obmp_packet_sender.cpp:472) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=27] dump txn free route audit_record(value=4, session->get_sessid()=3222118872, session->get_proxy_sessid()=720999913969918232)
[2023-07-23 11:04:40.482416] WDIAG [SERVER] force_disconnect (obmp_packet_sender.cpp:732) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4000] force disconnect(lbt()="0xdaaa2a5 0x7622c59 0x75ff7f8 0x3a1466e 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.482439] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:746) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=16][errcode=0] server close connection(sessid=3222118872, proxy_sessid=720999913969918232, stack="0xdaaa2a5 0x7622d92 0x75ff639 0x75ff800 0x3a1466e 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.482454] WDIAG [SERVER] process (obmp_query.cpp:424) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=13][errcode=-5066] disconnect connection(ret=-5066, ret="OB_ERR_SESSION_INTERRUPTED")
[2023-07-23 11:04:40.482469] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:152) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=11][errcode=-4009] sock has error(ret=-4009)
[2023-07-23 11:04:40.482478] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:929) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=10][errcode=-4009] write response fail(ret=-4009)
[2023-07-23 11:04:40.482485] WDIAG [SERVER] process (obmp_query.cpp:436) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=6][errcode=-5066] failed to flush_buffer(tmp_ret=-4009)
[2023-07-23 11:04:40.482496] TRACE [TRACE] after_process (obmp_base.cpp:142) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FA-0-0] [lt=9] [slow query](TRACE=begin_ts=1690081480482243 2023-07-23 03:04:40.482243|[process_begin] u=0 in_queue_time:7511495, receive_ts:1690081472970748, enqueue_ts:1690081480482231|[start_sql] u=1 addr:{ip:"10.1.129.154", port:54282}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FA-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222118872, tenant_id:1002|[query_end] u=23 |[session] u=61 sid:3222118872, tenant_id:1002|[session] u=119 sid:3222118872, tenant_id:1002|[process_end] u=43 run_ts:1690081480482247|total_timeu=251)
[2023-07-23 11:04:40.483059] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.483172] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480482212 2023-07-23 03:04:40.482212|[process_begin] u=0 in_queue_time:4458386, receive_ts:1690081476023826, enqueue_ts:1690081480482202|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=40 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=5 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[get_das_id_begin] u=4 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=32 |[calc_partition_location_begin] u=688 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=10 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=53 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=6 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=1 |[close_plan_begin] u=30 |[close_das_task_begin] u=1 |[close_das_task_end] u=33 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=11 |[process_end] u=1 run_ts:1690081480482215|total_timeu=958)
[2023-07-23 11:04:40.484215] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480392532, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:133, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.484273] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83127][T1002_LSMetaCh][T1001][YB420A01819C-0005FDE7212A8A57-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.484284] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83127][T1002_LSMetaCh][T1001][YB420A01819C-0005FDE7212A8A57-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.484288] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=1] will sleep(sleep_us=100000, remain_us=14685560, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=130, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:40.487195] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83364][T1001_LSMetaCh][T1][YB420A01819C-0005FDE7213A8991-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.487210] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83364][T1001_LSMetaCh][T1][YB420A01819C-0005FDE7213A8991-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.489729] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3328][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=1/1, request done=303730841/342494226, request doing=0/0)
[2023-07-23 11:04:40.490187] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3335][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/1, request done=303731088/342537638, request doing=0/0)
[2023-07-23 11:04:40.490210] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.490252] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3326][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=1/1, request done=303730635/343318437, request doing=0/0)
[2023-07-23 11:04:40.490301] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3330][RpcIO][T0][Y0-0000000000000000-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=1/1, request done=303731294/343280498, request doing=1/0)
[2023-07-23 11:04:40.490353] INFO  [CLOG] handle_sync_request (log_request_handler.cpp:486) [83649][T1002_L0_G1][T1002][YB420A01819C-0005FDE7216A81B6-0-0] [lt=32] i am not leader(ret=-4038, palf_id=1003, req={get_type:1}, role=2, is_pending_state=true)
[2023-07-23 11:04:40.490388] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3322][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/1, request done=303731512/343004008, request doing=0/0)
[2023-07-23 11:04:40.490422] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3333][RpcIO][T0][Y0-0000000000000000-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=1/1, request done=303731072/343732281, request doing=0/0)
[2023-07-23 11:04:40.490465] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3336][RpcIO][T0][YB420A01819C-0005FDE72E6CB09F-0-0] [lt=9] [RPC EASY STAT](log_str=conn count=1/1, request done=303730937/343748762, request doing=0/0)
[2023-07-23 11:04:40.490524] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3331][RpcIO][T0][YB420A01819C-0005FDE72B4C5444-0-0] [lt=8] [RPC EASY STAT](log_str=conn count=1/1, request done=303731621/342421206, request doing=0/0)
[2023-07-23 11:04:40.490558] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3334][RpcIO][T0][YB420A01819C-0005FDE72DCBDECF-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=1/1, request done=303731089/342184263, request doing=0/0)
[2023-07-23 11:04:40.490569] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3325][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=1/1, request done=303731160/343565042, request doing=0/0)
[2023-07-23 11:04:40.490767] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3332][RpcIO][T0][YB420A01819C-0005FDE72B4C543D-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=1/1, request done=303730946/343464969, request doing=0/0)
[2023-07-23 11:04:40.490902] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C25-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.490916] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C25-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.491000] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3337][RpcIO][T0][YB420A01819C-0005FDE72EAFC4F7-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=1/1, request done=303731290/343247566, request doing=0/0)
[2023-07-23 11:04:40.492789] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.492800] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.492827] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=14977272, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.492838] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.492854] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] get wrs ts(ls_id={id:1001}, delta=15085719, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.492855] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.492862] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.492876] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1002}, delta=15415006, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.492886] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.492887] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=15085957, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.492899] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1003}, delta=15349147, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.492907] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.492900] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.493304] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C33-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.493322] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C33-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.494201] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.494248] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.494256] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.494269] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.494275] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.494518] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.494529] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.494542] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.494564] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=21][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.494582] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=16] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.494601] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=16] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.494608] WDIAG [SHARE] renew_master_rootserver (ob_rs_mgr.cpp:355) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4638] no leader finded(ret=-4638, ret="OB_RS_NOT_MASTER", leader_exist=false, ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.494625] INFO  [SHARE] renew_master_rootserver (ob_rs_mgr.cpp:366) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=17] [RS_MGR] new master rootserver found(rootservice="0.0.0.0:0", cluster_id=3)
[2023-07-23 11:04:40.494633] WDIAG [SHARE] renew_master_rootserver (ob_rs_mgr.cpp:311) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4638] failed to renew master rootserver(ret=-4638, ret="OB_RS_NOT_MASTER")
[2023-07-23 11:04:40.494640] WDIAG [SERVER] renew_lease (ob_lease_state_mgr.cpp:263) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4638] renew_master_rootserver failed(ret=-4638)
[2023-07-23 11:04:40.494657] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.494663] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.494673] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.494679] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.494817] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.494826] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.494839] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.494856] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.494873] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=16] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.494890] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=15] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.494898] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480494896, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.494915] TRACE [TRACE] renew_lease (ob_lease_state_mgr.cpp:286) [3779][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=16] [slow heartbeat](TRACE=begin_ts=1690081480494184 2023-07-23 03:04:40.494184|[renew_lease_begin] u=0 |[do_renew_lease_begin] u=1 |[send_heartbeat_begin] u=11 |[send_heartbeat_end] u=17 ret:-4122|[do_renew_lease_end] u=1 ret:-4122|[renew_master_rs_begin] u=1 |[post_packet] u=31 ret:-4122, pcode:330, addr:{ip:"10.1.129.154", port:1001}|[post_packet] u=22 ret:-4122, pcode:330, addr:{ip:"10.1.129.155", port:1001}|[post_packet] u=35 ret:0, pcode:330, addr:{ip:"192.168.1.1", port:1001}|[post_packet] u=353 ret:-4122, pcode:330, addr:{ip:"10.1.129.154", port:1001}|[post_packet] u=16 ret:-4122, pcode:330, addr:{ip:"10.1.129.155", port:1001}|[post_packet] u=23 ret:0, pcode:330, addr:{ip:"192.168.1.1", port:1001}|[renew_lease_end] u=219 ret:-4638|total_timeu=730)
[2023-07-23 11:04:40.495344] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=10] new mysql sessid created(conn.sessid_=3222085985, support_ssl=false)
[2023-07-23 11:04:40.495364] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=9] sm conn init succ(conn.sessid_=3222085985, sess.client_addr_="192.168.1.1:43084")
[2023-07-23 11:04:40.495372] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3392][sql_nio14][T0][Y0-0000000000000000-0-0] [lt=7] accept one succ(*s={this:0x7fd420e4aab0, fd:390, err:0, last_decode_time_:0, last_write_time_:1690081480495363, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:40.495949] INFO  [SERVER] process (obmp_connect.cpp:369) [3194][MysqlQueueTh2][T1][Y0-0005FDE6FC4C4544-0-0] [lt=17] MySQL LOGIN(direct_client_ip="192.168.1.1", client_ip=192.168.1.1, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3222085985, proxy_sessid=720999922559852115, sess_create_time=0, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=129887, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:40.496188] INFO  [SERVER] process (obmp_init_db.cpp:53) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C85-0-0] [lt=10] init db(db_name=oceanbase)
[2023-07-23 11:04:40.496246] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C85-0-0] [lt=20] success add system var to ok pack(str_kv={key:"character_set_database", value:"45"}, change_var={id_:5, old_val_:{"BIGINT":45}}, new_val={"BIGINT":45}, session.get_sessid()=3222085985, session.get_proxy_sessid()=720999922559852115, change_var.id_=5)
[2023-07-23 11:04:40.496276] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C85-0-0] [lt=29] success add system var to ok pack(str_kv={key:"collation_database", value:"45"}, change_var={id_:10, old_val_:{"BIGINT":45}}, new_val={"BIGINT":45}, session.get_sessid()=3222085985, session.get_proxy_sessid()=720999922559852115, change_var.id_=10)
[2023-07-23 11:04:40.496724] INFO  [SERVER] add_changed_session_info (obmp_utils.cpp:81) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C86-0-0] [lt=36] success add system var to ok pack(str_kv={key:"ob_enable_transmission_checksum", value:"1"}, change_var={id_:10035, old_val_:{"BIGINT":1}}, new_val={"BIGINT":1}, session.get_sessid()=3222085985, session.get_proxy_sessid()=720999922559852115, change_var.id_=10035)
[2023-07-23 11:04:40.496988] INFO  [SQL.OPT] weak_select_replicas (ob_log_plan.cpp:2265) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=25] selected replica (intersect_server_list=[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:0, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686539761088454, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}{ls_replica_location:{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, attr_:{pos_type_:1, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686281947823862, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}], 
 phy_tbl_loc_info_list=[{table_location_key:18446744073709551605, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:0, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686539761088454, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}, {ls_replica_location:{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, attr_:{pos_type_:1, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686281947823862, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480494896}, selected_replica_idx:0, priority_replica_idxs:[0, 1]}], duplicate_type:0}, {table_location_key:18446744073709551607, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:0, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686539761088454, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}, {ls_replica_location:{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, attr_:{pos_type_:1, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686281947823862, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480494896}, selected_replica_idx:0, priority_replica_idxs:[0, 1]}], duplicate_type:0}, {table_location_key:18446744073709551609, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:0, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686539761088454, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}, {ls_replica_location:{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, attr_:{pos_type_:1, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686281947823862, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480494896}, selected_replica_idx:0, priority_replica_idxs:[0, 1]}], duplicate_type:0}, {table_location_key:18446744073709551613, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:0, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686539761088454, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}, {ls_replica_location:{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, attr_:{pos_type_:1, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686281947823862, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480494896}, selected_replica_idx:0, priority_replica_idxs:[0, 1]}], duplicate_type:0}, {table_location_key:18446744073709551611, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:0, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686539761088454, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}, {ls_replica_location:{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, attr_:{pos_type_:1, merge_status_:1, zone_type_:0, zone_status_:2, start_service_time_:1686281947823862, server_stop_time_:0, server_status_:1}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480494896}, selected_replica_idx:0, priority_replica_idxs:[0, 1]}], duplicate_type:0}], 
 route_policy={local_addr_:"192.168.1.1:1001", local_locality_:{inited:true, addr:"192.168.1.1:1001", zone:"zone3", zone_type:0, idc:"home_zb", region:"datazone", is_active:true, start_service_time:1686539761088454, server_stop_time:0, server_status:1}, server_locality_array_:[{inited:true, addr:"10.1.129.154:1001", zone:"zone1", zone_type:0, idc:"home_zb", region:"datazone", is_active:true, start_service_time:1686281947823862, server_stop_time:0, server_status:1}, {inited:true, addr:"10.1.129.155:1001", zone:"zone2", zone_type:0, idc:"home_zb", region:"datazone", is_active:true, start_service_time:1686281959362232, server_stop_time:0, server_status:1}, {inited:true, addr:"192.168.1.1:1001", zone:"zone3", zone_type:0, idc:"home_zb", region:"datazone", is_active:true, start_service_time:1686539761088454, server_stop_time:0, server_status:1}], has_refresh_locality_:false, has_readonly_zone_:false, is_inited_:true}, 
 route_policy_ctx={policy_type_:1, consistency_level_:2, is_proxy_priority_hit_support_:false, tenant_id_:1, max_read_stale_time_:5000000}, 
 follower_first_feedback=0, 
 is_hit_partition=true)
[2023-07-23 11:04:40.497269] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=246][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.497305] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.497605] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83344][T1001_FreInfoRe][T1001][YB420A01819C-0005FDE72B8C636A-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.497620] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83344][T1001_FreInfoRe][T1001][YB420A01819C-0005FDE72B8C636A-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.498095] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83344][T1001_FreInfoRe][T1001][YB420A01819C-0005FDE72B8C636B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.498635] INFO  [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1331) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=4] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1001)
[2023-07-23 11:04:40.498650] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1022) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1001, mem_hold=98566144, mem_limit=118111600, cache_obj_num=285, cache_node_num=285)
[2023-07-23 11:04:40.498659] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1039) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=9] end lib cache evict(tenant_id=1001, cache_evict_num=0, mem_hold=98566144, mem_limit=118111600, cache_obj_num=285, cache_node_num=285)
[2023-07-23 11:04:40.498668] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2024) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=7] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:40.501680] INFO  [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1799) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=7] Dumping All Cache Objs(alloc_obj_list.count()=285, alloc_obj_list=[{obj_id:20, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109179}, {obj_id:154, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116796}, {obj_id:195, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106976}, {obj_id:196, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129603}, {obj_id:197, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117342}, {obj_id:200, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115569}, {obj_id:201, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115590}, {obj_id:210, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117618}, {obj_id:213, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110485}, {obj_id:216, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109440}, {obj_id:217, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125156}, {obj_id:226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108544}, {obj_id:227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115837}, {obj_id:229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108552}, {obj_id:232, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109728}, {obj_id:234, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111259}, {obj_id:385, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88881}, {obj_id:390, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:11552}, {obj_id:391, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:916091}, {obj_id:392, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106461}, {obj_id:393, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134402}, {obj_id:394, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137472}, {obj_id:395, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132350}, {obj_id:399, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:400, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107490}, {obj_id:401, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110398}, {obj_id:403, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124488}, {obj_id:404, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121664}, {obj_id:409, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175250}, {obj_id:413, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156033}, {obj_id:414, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145796}, {obj_id:421, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144114}, {obj_id:422, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131804}, {obj_id:423, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167182}, {obj_id:425, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236458}, {obj_id:427, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219915}, {obj_id:432, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143476}, {obj_id:433, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164310}, {obj_id:434, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220663}, {obj_id:435, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203920}, {obj_id:439, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153942}, {obj_id:440, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132418}, {obj_id:441, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213169}, {obj_id:442, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176390}, {obj_id:443, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:428668}, {obj_id:444, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208446}, {obj_id:445, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:420917}, {obj_id:449, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143185}, {obj_id:450, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131501}, {obj_id:451, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162857}, {obj_id:452, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:397356}, {obj_id:453, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216472}, {obj_id:454, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:471683}, {obj_id:455, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199451}, {obj_id:458, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145055}, {obj_id:459, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132111}, {obj_id:460, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169934}, {obj_id:462, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249916}, {obj_id:464, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240171}, {obj_id:467, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147492}, {obj_id:468, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182923}, {obj_id:470, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:299640}, {obj_id:472, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286401}, {obj_id:476, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132741}, {obj_id:477, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171395}, {obj_id:478, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:154355}, {obj_id:479, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251338}, {obj_id:480, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182132}, {obj_id:483, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145342}, {obj_id:484, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172824}, {obj_id:485, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135088}, {obj_id:486, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257151}, {obj_id:487, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159098}, {obj_id:488, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242027}, {obj_id:491, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:148679}, {obj_id:492, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175363}, {obj_id:496, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143827}, {obj_id:497, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165757}, {obj_id:499, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226625}, {obj_id:500, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210208}, {obj_id:503, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:479760}, {obj_id:507, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123880}, {obj_id:512, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144720}, {obj_id:513, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132287}, {obj_id:514, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155764}, {obj_id:515, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229546}, {obj_id:519, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140461}, {obj_id:520, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165558}, {obj_id:523, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145927}, {obj_id:1085, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172321}, {obj_id:1088, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:496026}, {obj_id:1089, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:590667}, {obj_id:1108, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:571203}, {obj_id:1534, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111508}, {obj_id:1720, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110232}, {obj_id:1721, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108568}, {obj_id:2813, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:904134}, {obj_id:2814, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1074639}, {obj_id:2824, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126601}, {obj_id:2825, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149042}, {obj_id:2828, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196135}, {obj_id:2829, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225299}, {obj_id:2900, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:382028}, {obj_id:2904, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:504810}, {obj_id:2910, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160620}, {obj_id:2914, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257326}, {obj_id:2927, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159996}, {obj_id:2943, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:320727}, {obj_id:2944, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:398335}, {obj_id:2945, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213512}, {obj_id:2946, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:232858}, {obj_id:2947, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231038}, {obj_id:2948, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:259955}, {obj_id:2955, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215695}, {obj_id:2956, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236436}, {obj_id:2977, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:364316}, {obj_id:2979, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:476386}, {obj_id:2986, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171290}, {obj_id:2987, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173475}, {obj_id:2991, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160324}, {obj_id:2994, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150077}, {obj_id:3004, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1077957}, {obj_id:3018, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193338}, {obj_id:3019, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215348}, {obj_id:3088, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:373210}, {obj_id:3091, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:491688}, {obj_id:3111, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228603}, {obj_id:3183, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:375409}, {obj_id:3187, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:495398}, {obj_id:3192, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114403}, {obj_id:3194, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132114}, {obj_id:3196, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142870}, {obj_id:3197, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161389}, {obj_id:3198, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212913}, {obj_id:3199, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208931}, {obj_id:3200, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245295}, {obj_id:3201, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191638}, {obj_id:3203, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133322}, {obj_id:3206, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143237}, {obj_id:3208, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136758}, {obj_id:3212, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132420}, {obj_id:3215, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133954}, {obj_id:3217, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134278}, {obj_id:3219, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134586}, {obj_id:3224, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132714}, {obj_id:3226, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130839}, {obj_id:3227, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228133}, {obj_id:3229, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133030}, {obj_id:3233, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221640}, {obj_id:3260, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241140}, {obj_id:3279, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:261670}, {obj_id:3280, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:307076}, {obj_id:3344, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211307}, {obj_id:3346, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229535}, {obj_id:3354, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198169}, {obj_id:3355, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209793}, {obj_id:3364, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371003}, {obj_id:3367, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:488294}, {obj_id:3386, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161936}, {obj_id:3417, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:259444}, {obj_id:3418, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:302716}, {obj_id:3421, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228841}, {obj_id:3423, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272560}, {obj_id:3424, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:323080}, {obj_id:3425, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233184}, {obj_id:3427, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263482}, {obj_id:3429, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:266008}, {obj_id:3430, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:313559}, {obj_id:3431, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257249}, {obj_id:3432, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:302583}, {obj_id:3435, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209137}, {obj_id:3436, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226426}, {obj_id:3443, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:377617}, {obj_id:3446, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:498848}, {obj_id:3450, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162941}, {obj_id:3452, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120793}, {obj_id:3453, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125150}, {obj_id:3454, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106336}, {obj_id:3455, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166062}, {obj_id:3456, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164083}, {obj_id:3457, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173024}, {obj_id:3458, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176583}, {obj_id:3459, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167827}, {obj_id:3460, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167224}, {obj_id:3461, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250859}, {obj_id:3476, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163876}, {obj_id:3494, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:311929}, {obj_id:3495, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:382142}, {obj_id:3502, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193818}, {obj_id:3503, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202903}, {obj_id:3504, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:305358}, {obj_id:3505, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:373402}, {obj_id:3512, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290500}, {obj_id:3513, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:371313}, {obj_id:3526, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237867}, {obj_id:3527, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137034}, {obj_id:3529, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149414}, {obj_id:3530, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155318}, {obj_id:3531, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149088}, {obj_id:3532, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150552}, {obj_id:3533, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158615}, {obj_id:3534, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109997}, {obj_id:3535, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218405}, {obj_id:3573, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204758}, {obj_id:3574, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219421}, {obj_id:3579, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202572}, {obj_id:3580, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216139}, {obj_id:3597, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231390}, {obj_id:3640, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241951}, {obj_id:3641, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276520}, {obj_id:3646, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200355}, {obj_id:3647, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212858}, {obj_id:3648, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250703}, {obj_id:3649, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290613}, {obj_id:3650, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:318466}, {obj_id:3651, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:392907}, {obj_id:3691, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234643}, {obj_id:3692, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169830}, {obj_id:3693, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:166691}, {obj_id:3694, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176446}, {obj_id:3695, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177696}, {obj_id:3700, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185061}, {obj_id:3701, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189642}, {obj_id:3702, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182888}, {obj_id:3703, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186225}, {obj_id:3704, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206937}, {obj_id:3705, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222717}, {obj_id:3706, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189429}, {obj_id:3707, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174138}, {obj_id:3708, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196108}, {obj_id:3709, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173051}, {obj_id:3710, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196007}, {obj_id:3711, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206080}, {obj_id:3712, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:355688}, {obj_id:3713, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:309741}, {obj_id:3714, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:451053}, {obj_id:3715, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:379695}, {obj_id:3716, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191644}, {obj_id:3717, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199480}, {obj_id:3718, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180699}, {obj_id:3719, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183816}, {obj_id:3720, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178489}, {obj_id:3721, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179453}, {obj_id:3722, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187243}, {obj_id:3723, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192970}, {obj_id:3724, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:307577}, {obj_id:3725, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:376128}, {obj_id:3726, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235392}, {obj_id:3727, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:266803}, {obj_id:3728, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:294863}, {obj_id:3729, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:378201}, {obj_id:3730, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:385496}, {obj_id:3733, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227367}, {obj_id:3735, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:519272}, {obj_id:3738, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264456}, {obj_id:3740, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215149}, {obj_id:3741, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117694}, {obj_id:3742, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211909}, {obj_id:3743, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124622}, {obj_id:3744, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128288}, {obj_id:3745, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110732}, {obj_id:3746, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122619}, {obj_id:3747, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122641}, {obj_id:3748, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118086}, {obj_id:3749, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110316}, {obj_id:3750, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:121815}, {obj_id:3751, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125255}, {obj_id:3752, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112634}, {obj_id:3753, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119794}, {obj_id:3754, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176518}, {obj_id:3755, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164881}, {obj_id:3756, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113232}, {obj_id:3757, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142629}, {obj_id:3758, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121301}, {obj_id:3759, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112702}, {obj_id:3760, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129293}, {obj_id:3761, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201012}, {obj_id:3762, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137606}, {obj_id:3763, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136771}, {obj_id:3764, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167626}, {obj_id:3766, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163100}, {obj_id:3768, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122303}, {obj_id:3769, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130157}, {obj_id:3770, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172025}, {obj_id:3771, tenant_id:1001, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170852}])
[2023-07-23 11:04:40.502209] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2032) [83289][T1001_PlanCache][T1001][Y0-0000000000000000-0-0] [lt=528] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:40.502507] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.502566] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480502399 2023-07-23 03:04:40.502399|[process_begin] u=0 in_queue_time:5277708, receive_ts:1690081475224690, enqueue_ts:1690081480502388|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=5 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=38 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=3 |[parse_job_end] u=5 |[remote_task_completed] u=59 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29257}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=15 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=2 |[close_plan_begin] u=2 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=13 |[process_end] u=1 run_ts:1690081480502405|total_timeu=165)
[2023-07-23 11:04:40.512705] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196772}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480400896, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.512752] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=47] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196772}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480400896, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.512793] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=39] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480494567, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196772}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480400896, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.512853] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480512514 2023-07-23 03:04:40.512514|[process_begin] u=0 in_queue_time:7852379, receive_ts:1690081472660135, enqueue_ts:1690081480512503|[start_sql] u=1 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=74 |[check_priv] u=11 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=4 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=4 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=4 |[value_after_calc_row] u=9 |[insert_start] u=1 |[get_das_id_begin] u=17 |[get_das_id_end] u=0 |[insert_end] u=4 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=12 |[sync_auto_value] u=4 |[close_plan_begin] u=29 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=131 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=3 |[query_end] u=14 |[process_end] u=1 run_ts:1690081480512517|total_timeu=337)
[2023-07-23 11:04:40.513667] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879364-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.514671] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879364-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.515641] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879364-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.516610] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879364-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.518884] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.530677] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.535795] INFO  [SQL.PC] inner_cache_evict (ob_ps_cache.cpp:796) [66390][T1002_PsCacheEv][T1002][Y0-0000000000000000-0-0] [lt=14] ps cache evict(tenant_id=1002, stmt_id_map_.size()=120, stmt_info_map_.size()=120, op.get_used_size()=217964, get_mem_high()=478351980, expired_stmt_ids.count()=0, closed_stmt_ids.count()=120)
[2023-07-23 11:04:40.536150] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=1] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480474596, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908684}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480479824, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.536220] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=0] will sleep(sleep_us=95000, remain_us=20140318, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=95, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:40.548118] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.548145] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480548111]})
[2023-07-23 11:04:40.548168] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480548159]})
[2023-07-23 11:04:40.548193] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.548210] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=16][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.548236] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.548246] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.548417] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=14][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.548435] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=17][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.548440] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480548429]})
[2023-07-23 11:04:40.548456] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=13][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.548494] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=37][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.548539] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=43] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.548574] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=31] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.548589] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B064-0-0] [lt=10] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480548586, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.548770] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1FC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.549010] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C41-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.549027] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C41-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.552923] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=45] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480552915})
[2023-07-23 11:04:40.552935] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC3E-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.552962] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3291][][T0][YB420A01819C-0005FDE715FDDC3E-0-0] [lt=19][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.553605] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.553593] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0] start stmt(ret=-4283, auto_commit=true, session_id=3222085985, snapshot={this:0x7fd52074af38, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=0, tx_desc={this:0x7fd4e3f513b0, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:3222085985, assoc_session_id:3222085985, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:1, alloc_ts:1690081480496674, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:-1, cluster_version:0, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, plan_type=1, stmt_type=1, has_for_update=false, query_start_time=1690081480496875, use_das=false, nested_level=0, session={this:0x7fd4fee97fa8, id:3222085985, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"proxyro@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4e3f513b0}, plan=0x7fd4ca5ce050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.553674] TRACE [TRACE] after_process (obmp_base.cpp:156) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480496893 2023-07-23 03:04:40.496893|[process_begin] u=0 in_queue_time:17, receive_ts:1690081480496875, enqueue_ts:1690081480496875|[start_sql] u=0 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=45 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=5 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=3 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=282 |[pc_choose_plan] u=2 |[check_priv] u=5 |[plan_id] u=1 plan_id:18469|[do_open_plan_begin] u=3 plan_id:18469|[sql_start_stmt_begin] u=1 |[post_packet] u=34 ret:-4122, pcode:1808, addr:{ip:"10.1.129.154", port:1001}|[sql_start_stmt_end] u=56368 |[do_open_plan_end] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=4 |[query_end] u=8 |[process_end] u=1 run_ts:1690081480496896|total_timeu=56779)
[2023-07-23 11:04:40.553723] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480552839 2023-07-23 03:04:40.552839|[process_begin] u=0 in_queue_time:4599519, receive_ts:1690081475953320, enqueue_ts:1690081480552830|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=59 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=32 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=4 |[plan_id] u=0 plan_id:20478|[do_open_plan_begin] u=3 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=15 |[do_open_plan_end] u=0 |[get_das_id_begin] u=36 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=21 |[calc_partition_location_begin] u=538 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=8 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=35 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=5 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=28 |[close_das_task_begin] u=1 |[close_das_task_end] u=25 |[start_end_stmt] u=20 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=16 |[process_end] u=1 run_ts:1690081480552842|total_timeu=883)
[2023-07-23 11:04:40.553852] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45605][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=73][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.553866] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45605][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.553911] TRACE [TRACE] after_process (obmp_base.cpp:156) [45605][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480553736 2023-07-23 03:04:40.553736|[process_begin] u=0 in_queue_time:56861, receive_ts:1690081480496875, enqueue_ts:1690081480553718|[start_sql] u=1 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=5 |[session] u=2 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=38 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=2 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=4 |[pc_choose_plan] u=2 |[check_priv] u=5 |[plan_id] u=0 plan_id:18472|[do_open_plan_begin] u=2 plan_id:18472|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[parse_job_begin] u=3 |[parse_job_end] u=7 |[remote_task_completed] u=63 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29259}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=11 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=2 |[close_plan_begin] u=3 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=11 |[process_end] u=1 run_ts:1690081480553743|total_timeu=174)
[2023-07-23 11:04:40.553956] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:267) [45605][T1_L0_G100][T1][Y0-0000000000000000-0-0] [lt=42][errcode=0] tenant push retry request to wait queue(tenant=1, req={packet:{header:{length:158, sequence:0}, can_reroute:false, weak_read:true}, type:1, group:0, sql_req_level:0, connection_phase:2, recv_timestamp_:1690081480496875, enqueue_timestamp_:1690081480553718, request_arrival_time_:0, trace_id_:YB420A01819C-0005FDE72D1C9C87-0-0})
[2023-07-23 11:04:40.557359] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.557382] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15095433, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.557392] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.559282] INFO  [OCCAM] ObOccamTimerTaskRAIIHandle (ob_occam_timer.h:363) [3625][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=43] task handle constructed(*this={this:0x7fd54034b9e0, task:NULL, is_inited:false, is_running:false})
[2023-07-23 11:04:40.560116] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:644) [3308][Occam][T1][YB420A01819C-0005FDE625B7976C-0-0] [lt=0] inner session no need register to deadlock(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4122, exec_ctx.get_errcode()="OB_RPC_POST_ERROR", session={this:0x7fd4f5a6a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fd5000}, desc=NULL, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.560146] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [3308][Occam][T1][YB420A01819C-0005FDE625B7976C-0-0] [lt=31] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4122, exec_ctx.get_errcode()="OB_RPC_POST_ERROR", session={this:0x7fd4f5a6a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fd5000}, desc=NULL, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.560172] INFO  [STORAGE.TRANS] rollback_to_global_implicit_savepoint_ (ob_tx_api.cpp:1023) [3308][Occam][T1][YB420A01819C-0005FDE625B7976C-0-0] [lt=23] add tx part with extra_touched_ls(extra_touched_ls=[{id:1}], tx.tx_id={txid:44908685})
[2023-07-23 11:04:40.560196] INFO  [STORAGE.TRANS] abort_tx_ (ob_trans_service_v4.cpp:575) [3308][Occam][T1][YB420A01819C-0005FDE625B7976C-0-0] [lt=0] abort tx(ret=0, *this={is_inited_:true, tenant_id_:1, this:0x7fd550e04030}, tx={this:0x7fd4f4fd5000, tx_id:{txid:44908685}, state:6, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081480558731, op_sn:3, alloc_ts:1690081480559782, active_ts:1690081480559782, commit_ts:-1, finish_ts:-1, timeout_us:29999989, lock_timeout_us:-1, expire_ts:1690081510559771, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"0.0.0.0:0", epoch:-1, first_scn:9223372036854775807, last_scn:9223372036854775807, last_touch_ts:1}], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:4, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, cause=4)
[2023-07-23 11:04:40.560241] INFO  [STORAGE.TRANS] start_epoch_ (ob_tx_api.cpp:1736) [3308][Occam][T1][YB420A01819C-0005FDE625B7976C-0-0] [lt=44] tx start new epoch(ret=0, tx={this:0x7fd4f4fd5000, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:0, op_sn:4, alloc_ts:1690081480559782, active_ts:0, commit_ts:0, finish_ts:0, timeout_us:0, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1})
[2023-07-23 11:04:40.560273] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [3308][Occam][T1][YB420A01819C-0005FDE625B7976C-0-0] [lt=29] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480558731, tx_desc={this:0x7fd4f4fd5000, tx_id:{txid:0}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:0, op_sn:4, alloc_ts:1690081480559782, active_ts:0, commit_ts:0, finish_ts:0, timeout_us:0, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[{id:1}], cflict_txs:[]}, rollback=true, session={this:0x7fd4f5a6a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f4fd5000}, exec_ctx.get_errcode()=-4122)
[2023-07-23 11:04:40.560344] INFO  [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1915) [3308][Occam][T0][Y0-0000000000000000-0-0] [lt=1] execute write sql(ret=-4122, tenant_id=1, affected_rows=0, sql=INSERT INTO __all_server_event_history (gmt_create, module, event, name1, value1, name2, value2, name3, value3, name4, value4, svr_ip, svr_port) VALUES (usec_to_time(1690081480559334), 'ELECTION', 'vote', 'TENANT_ID', 1, 'LS_ID', 1, 'INFO', 'vote for "192.168.1.1:1001", reason:the only request', 'HAPPENED_TIME', '2023-07-23 11:04:40.559278', '192.168.1.1', 2882))
[2023-07-23 11:04:40.561658] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1954-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.561670] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1954-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.568225] WDIAG [SHARE] refresh (ob_task_define.cpp:382) [3138][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=15][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4283, dropped:126, tid:83045}])
[2023-07-23 11:04:40.573179] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196773}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480461991, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.573216] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=37] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196773}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480461991, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.573248] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=30] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480558732, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196773}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480461991, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.573289] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480573034 2023-07-23 03:04:40.573034|[process_begin] u=0 in_queue_time:7297794, receive_ts:1690081473275239, enqueue_ts:1690081480573021|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=50 |[check_priv] u=9 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=8 |[insert_start] u=1 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=10 |[sync_auto_value] u=2 |[close_plan_begin] u=25 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=97 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=2 |[query_end] u=11 |[process_end] u=1 run_ts:1690081480573036|total_timeu=254)
[2023-07-23 11:04:40.573553] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079327-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.573920] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079327-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.574261] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079327-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.574609] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079327-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.579372] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B85-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.579387] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B85-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.583681] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.583795] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480583103 2023-07-23 03:04:40.583103|[process_begin] u=0 in_queue_time:4559276, receive_ts:1690081476023826, enqueue_ts:1690081480583088|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=30 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=19 |[calc_partition_location_begin] u=466 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=6 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=38 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=57 |[close_das_task_begin] u=0 |[close_das_task_end] u=24 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[query_end] u=8 |[process_end] u=1 run_ts:1690081480583105|total_timeu=691)
[2023-07-23 11:04:40.584372] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=24][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:130, local_retry_times:130, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.584386] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=13][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.584390] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.584393] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.584404] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.584444] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.592825] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1}, delta=15077220, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.592845] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.592903] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.592874] INFO  [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1029) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19] get_min_unreplayed_log_info(lsn={lsn:704197620496}, scn={val:1690081465406750437}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:2, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1690081465556879, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:227740, ret_code_:0}}, next_to_submit_lsn_:{lsn:704197620496}, committed_end_lsn_:{lsn:704197620496}, next_to_submit_scn_:{val:1690081465406750437}, base_lsn_:{lsn:0}, base_scn_:{val:1686557628709509541}, iterator_:{iterator_impl:{buf_:0x7fd36fa05000, next_round_pread_size:2121728, curr_read_pos:122, curr_read_buf_start_pos:0, curr_read_buf_end_pos:122, log_storage_:{IteratorStorage:{start_lsn:{lsn:704197620374}, end_lsn:{lsn:704197620496}, read_buf:{buf_len_:2125824, buf_:0x7fd36fa05000}, block_size:67104768, log_storage_:0x7fd4653b6070}, IteratorStorageType::"DiskIteratorStorage"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1690081465406750436}, curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:34, scn_:{val:1690081465406750436}, data_checksum:154833137, flag:1}}, init_mode_version:0, accumlate_checksum:2486464045}}}})
[2023-07-23 11:04:40.592929] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15185905, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.592933] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=59] get wrs ts(ls_id={id:1001}, delta=15185667, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.592941] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.592944] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.592963] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get wrs ts(ls_id={id:1002}, delta=15514954, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.592975] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.592995] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get wrs ts(ls_id={id:1003}, delta=15449095, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.593007] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.593500] INFO  [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:374) [83111][T1002_TenantMet][T1002][Y0-0000000000000000-0-0] [lt=31] Recycle 0 table(ret=0, allocator_={used:5113134, total:9349053}, tablet_pool_={typeid(T).name():"N9oceanbase7storage8ObTabletE", sizeof(T):2560, used_obj_cnt:1357, free_obj_hold_cnt:15, allocator used:3600128, allocator total:3793664}, sstable_pool_={typeid(T).name():"N9oceanbase12blocksstable9ObSSTableE", sizeof(T):1088, used_obj_cnt:2848, free_obj_hold_cnt:2763, allocator used:6463872, allocator total:6737024}, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):3008, used_obj_cnt:0, free_obj_hold_cnt:4, allocator used:12288, allocator total:65408}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1920, used_obj_cnt:132, free_obj_hold_cnt:1213, allocator used:2668480, allocator total:2812544}, tablet count=1357, min_minor_cnt=118, pinned_tablet_cnt=0)
[2023-07-23 11:04:40.603129] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:351) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=3] ====== check clog disk timer task ======
[2023-07-23 11:04:40.603117] INFO  [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1171) [3286][DumpLSLoc][T0][YB420A01819C-0005FDE6258CF528-0-0] [lt=78] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480548586, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]}])
[2023-07-23 11:04:40.603145] INFO  [PALF] get_disk_usage (palf_env_impl.cpp:781) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=60825, used(MB):=48597)
[2023-07-23 11:04:40.603150] INFO  [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1171) [3286][DumpLSLoc][T0][YB420A01819C-0005FDE6258CF528-0-0] [lt=33] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:3}, renew_time:1690081465091764, replica_locations:[{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}, {server:"10.1.129.155:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}])
[2023-07-23 11:04:40.603178] INFO  [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1171) [3286][DumpLSLoc][T0][YB420A01819C-0005FDE6258CF528-0-0] [lt=26] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:3}, renew_time:1690081465093115, replica_locations:[{server:"10.1.129.154:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.1.129.155:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"192.168.1.1:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:54}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:3}, renew_time:1690081465093112, replica_locations:[{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}, {server:"10.1.129.155:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:3}, renew_time:1690081465093114, replica_locations:[{server:"10.1.129.154:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"10.1.129.155:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}, {server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:3}, renew_time:1690081465093109, replica_locations:[{server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}, {server:"10.1.129.155:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, {server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}]}])
[2023-07-23 11:04:40.603326] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=48][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586})
[2023-07-23 11:04:40.603347] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=21][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.603365] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=17][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.603374] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.603379] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:40.603407] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=27][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:40.603418] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:40.603424] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:40.603431] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:40.603954] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] succ to get rec scn(*this={this:0x7fd46b604030, ls_id:{id:1}, tenant_id:1002, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1690048802316336799}, last_push_gc_task_ts_:1690081221495637, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.603972] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=18] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc4080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1690048804114207}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc4080, snapshot_version:{val:1690048804114207}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.603994] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=17] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689738033637759567}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689738033637759567}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1})
[2023-07-23 11:04:40.604007] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd4f23e6250}, min_rec_scn={val:1690080742373882309}, ls_id_={id:1})
[2023-07-23 11:04:40.604479] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1690081457643664628})
[2023-07-23 11:04:40.604487] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=1, rec_log_ts={val:1690081371645352093})
[2023-07-23 11:04:40.604494] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080742373882309}, checkpoint_scn_in_ls_meta={val:1690080742373882309}, ls_id={id:1}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:40.605150] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:40.605202] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=21][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586})
[2023-07-23 11:04:40.605221] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=19][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.605239] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.605249] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.605254] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.605261] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:40.605266] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:40.605273] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.605280] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.605284] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.605300] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:40.605289] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x7fd46b686030, ls_id:{id:1001}, tenant_id:1002, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1689841617974500487}, last_push_gc_task_ts_:1689841618653371, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.605306] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:40.605311] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:40.605307] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=17] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc4d90, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1689841618655957}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc4d90, snapshot_version:{val:1689841618655957}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.605315] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:40.605319] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689736139869551129}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689736139869551129}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1001})
[2023-07-23 11:04:40.605322] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:40.605329] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:40.605329] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd465388250}, min_rec_scn={val:1690080741701524066}, ls_id_={id:1001})
[2023-07-23 11:04:40.605335] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:40.605339] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:40.605349] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:54, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:40.605359] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.605368] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=4619323)
[2023-07-23 11:04:40.605393] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949477097, cache_obj->added_lc()=false, cache_obj->get_object_id()=20503, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.605414] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10] [packet retry query](TRACE=begin_ts=1690081480603279 2023-07-23 03:04:40.603279|[process_begin] u=0 in_queue_time:5378588, receive_ts:1690081475224690, enqueue_ts:1690081480603270|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=4 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=37 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=105 |[transform_with_outline_begin] u=74 |[transform_with_outline_end] u=1 |[resolve_begin] u=2 |[resolve_end] u=572 |[transform_begin] u=23 |[optimize_begin] u=831 |[tl_calc_part_id_end] u=206 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=210 |[query_end] u=64 |[process_end] u=1 run_ts:1690081480603284|total_timeu=2134)
[2023-07-23 11:04:40.605669] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080741701524066}, checkpoint_scn_in_ls_meta={val:1690080741701524066}, ls_id={id:1001}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:40.606473] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] succ to get rec scn(*this={this:0x7fd46b708030, ls_id:{id:1002}, tenant_id:1002, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1689778308022019911}, last_push_gc_task_ts_:1689778309607868, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.606485] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc5aa0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1689778309609228}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc5aa0, snapshot_version:{val:1689778309609228}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.606498] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689738033632736044}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689738033632736044}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1002})
[2023-07-23 11:04:40.606508] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd45c21a250}, min_rec_scn={val:1690080741745971437}, ls_id_={id:1002})
[2023-07-23 11:04:40.606828] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080741745971437}, checkpoint_scn_in_ls_meta={val:1690080741745971437}, ls_id={id:1002}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:40.607630] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=8] succ to get rec scn(*this={this:0x7fd447204030, ls_id:{id:1003}, tenant_id:1002, state:"L_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1689838148069950986}, last_push_gc_task_ts_:1689838151799175, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.607642] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc67b0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1689838151801616}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc67b0, snapshot_version:{val:1689838151801616}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:40.607653] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689738033637759566}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689738033637759566}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1003})
[2023-07-23 11:04:40.607663] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd4653e4250}, min_rec_scn={val:1690080741730241082}, ls_id_={id:1003})
[2023-07-23 11:04:40.607989] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080741730241082}, checkpoint_scn_in_ls_meta={val:1690080741730241082}, ls_id={id:1003}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:40.607997] INFO  [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:264) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] cannot_recycle_log_size statistics(cannot_recycle_log_size=424670258, threshold=19134079303)
[2023-07-23 11:04:40.608009] INFO  [STORAGE] flush_if_need_ (ob_ls.cpp:1564) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] the ls no need flush to advance_checkpoint(get_ls_id()={id:1}, need_flush=false)
[2023-07-23 11:04:40.608018] INFO  [STORAGE] flush_if_need_ (ob_ls.cpp:1564) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=6] the ls no need flush to advance_checkpoint(get_ls_id()={id:1001}, need_flush=false)
[2023-07-23 11:04:40.608023] INFO  [STORAGE] flush_if_need_ (ob_ls.cpp:1564) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=3] the ls no need flush to advance_checkpoint(get_ls_id()={id:1002}, need_flush=false)
[2023-07-23 11:04:40.608030] INFO  [STORAGE] flush_if_need_ (ob_ls.cpp:1564) [83142][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=5] the ls no need flush to advance_checkpoint(get_ls_id()={id:1003}, need_flush=false)
[2023-07-23 11:04:40.608237] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [3734][T1_LSArchiveMgr][T1][YB420A01819C-0005FDE623878DB5-0-0] [lt=11] gc stale ls task succ
[2023-07-23 11:04:40.613418] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=23][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586})
[2023-07-23 11:04:40.613435] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=17][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551605, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.613451] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=15][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=5)
[2023-07-23 11:04:40.613459] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.613463] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551605, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551607, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551609, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551613, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551611, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551605, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551607, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551609, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551613, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551611, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:40.613537] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=73][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551605, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551607, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551609, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551613, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551611, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:40.613565] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=28][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:40.613570] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:40.613575] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:40.614214] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.614231] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=12][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.614248] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.614260] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.614271] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.618115] WDIAG [RPC.FRAME] create_session (ob_req_transport.cpp:263) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD8-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.618132] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD8-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.618492] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6] total rowcount, use statistics(table_meta_info_.table_row_count_=24, table_meta_info_.average_row_size_=1.170000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=2.808000000000000000e+03)
[2023-07-23 11:04:40.619602] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=20][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586})
[2023-07-23 11:04:40.619623] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=21][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480548586}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.619641] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=18][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.619647] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.619654] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.619661] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=8][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:40.619665] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:40.619674] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807)
[2023-07-23 11:04:40.619681] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.619685] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.619691] WDIAG [SQL.JO] generate_subquery_paths (ob_join_order.cpp:6498) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] failed to optimize sub-select(ret=-4722)
[2023-07-23 11:04:40.619695] WDIAG [SQL.JO] generate_normal_subquery_paths (ob_join_order.cpp:6466) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] failed to generate subquery path(ret=-4722)
[2023-07-23 11:04:40.619701] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807)
[2023-07-23 11:04:40.619708] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.619711] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.619717] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:40.619721] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=3][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:40.619725] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:40.619730] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:40.619739] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:40.619746] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:40.619753] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:40.619757] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] fail to handle text query(stmt=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807, ret=-4722)
[2023-07-23 11:04:40.619765] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:2, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:40.619775] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=9][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807)
[2023-07-23 11:04:40.619784] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=9877092)
[2023-07-23 11:04:40.619816] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949491520, cache_obj->added_lc()=false, cache_obj->get_object_id()=20504, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.619835] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=9] [packet retry query](TRACE=begin_ts=1690081480613376 2023-07-23 03:04:40.613376|[process_begin] u=0 in_queue_time:116501, receive_ts:1690081480496875, enqueue_ts:1690081480613368|[start_sql] u=0 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=4 |[session] u=1 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=28 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[pc_choose_plan] u=156 |[transform_with_outline_begin] u=61 |[transform_with_outline_end] u=1 |[resolve_begin] u=2 |[resolve_end] u=759 |[transform_begin] u=23 |[optimize_begin] u=3659 |[tl_calc_part_id_end] u=398 |[get_location_cache_begin] u=0 |[storage_estimation_begin] u=79 |[storage_estimation_end] u=1035 |[get_location_cache_begin] u=175 |[query_end] u=68 |[process_end] u=1 run_ts:1690081480613381|total_timeu=6458)
[2023-07-23 11:04:40.623632] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=1] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196774}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480512443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.623666] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=33] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196774}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480512443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.623691] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480558735, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196774}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480512443, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.623731] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480623496 2023-07-23 03:04:40.623496|[process_begin] u=0 in_queue_time:7963360, receive_ts:1690081472660135, enqueue_ts:1690081480623485|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=52 |[check_priv] u=8 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=2 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=9 |[sync_auto_value] u=3 |[close_plan_begin] u=19 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=0 |[end_stmt] u=88 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=10 |[process_end] u=1 run_ts:1690081480623499|total_timeu=234)
[2023-07-23 11:04:40.624103] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F9-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.624587] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F9-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.625045] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F9-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.625510] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793F9-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.630732] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.631301] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:95, local_retry_times:95, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.631337] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=25][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.631350] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.631361] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.631382] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.631482] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=1][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.635938] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73134578892, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:40.637366] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=20] Wash time detail, (compute_wash_size_time=99, refresh_score_time=832, wash_time=575)
[2023-07-23 11:04:40.640242] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.640253] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081480584438], now0=[mts=1690081480584438], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.640264] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:134, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.640289] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=24][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:40.640308] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=19][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:134, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.640332] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=22] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480558734, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:134, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.640367] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=35][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.640377] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.640384] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.640389] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.640396] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:40.640404] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=131, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:40.640414] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8] will sleep(sleep_us=100000, remain_us=14529434, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=131, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:40.646673] INFO  [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [3784][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=23] current monitor number(seq_=-1)
[2023-07-23 11:04:40.648997] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=1] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480648983]})
[2023-07-23 11:04:40.649020] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480649013]})
[2023-07-23 11:04:40.649051] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.649069] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=17][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.649092] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.649103] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=11][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.649222] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480649211]})
[2023-07-23 11:04:40.649280] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=15][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.649309] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=27][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.649333] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=13][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.649380] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=46][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.649415] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=32] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.649452] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=32] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.649480] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1FD-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.649469] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B065-0-0] [lt=11] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480649464, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.649681] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C26-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.653774] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=27] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480653767})
[2023-07-23 11:04:40.653782] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC3F-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.654131] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] Cache replace map node details(ret=0, replace_node_count=0, replace_time=11573, replace_start_pos=22700000, replace_num=100000)
[2023-07-23 11:04:40.654344] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.654450] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480653697 2023-07-23 03:04:40.653697|[process_begin] u=0 in_queue_time:4700376, receive_ts:1690081475953320, enqueue_ts:1690081480653684|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=55 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=27 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[exec_plan_end] u=6 |[do_open_plan_end] u=0 |[get_das_id_begin] u=38 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=15 |[calc_partition_location_begin] u=448 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=0 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:427|[do_open_plan_begin] u=7 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=32 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=23 |[close_das_task_begin] u=1 |[close_das_task_end] u=22 |[start_end_stmt] u=19 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=14 |[process_end] u=1 run_ts:1690081480653700|total_timeu=752)
[2023-07-23 11:04:40.657417] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.657441] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15195459, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.657452] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.659238] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=21858, clean_start_pos=9400000, clean_num=200000)
[2023-07-23 11:04:40.662593] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1955-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.668343] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1285) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS](tenant_id=1, io_status=group_id: 0, group_name: OTHER_GROUPS, mode:  read, size:    4096.00, iops:    11.99, rt:   113.17)
[2023-07-23 11:04:40.668354] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1307) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS](tenant_id=1, ref_cnt=2, io_config={group_num_:0, memory_limit_:17179869184, callback_thread_count_:8, unit_config_:{min_iops:10000, max_iops:10000, weight:1}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=100086048, callback_queues=[0, 0, 0, 0, 0, 0, 0, 0])
[2023-07-23 11:04:40.668373] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1285) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS](tenant_id=500, io_status=group_id: 0, group_name: OTHER_GROUPS, mode:  read, size: 2097152.00, iops:     2.00, rt:  3906.00)
[2023-07-23 11:04:40.668379] INFO  [COMMON] print_io_status (ob_io_manager.cpp:1307) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS](tenant_id=500, ref_cnt=2, io_config={group_num_:0, memory_limit_:536870912, callback_thread_count_:8, unit_config_:{min_iops:10000, max_iops:50000, weight:10000}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=37693728, callback_queues=[0, 0, 0, 0, 0, 0, 0, 0])
[2023-07-23 11:04:40.668395] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO SENDER STATUS](send_index=1, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668402] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO SENDER STATUS](send_index=2, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668406] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=3, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668411] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=4, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668415] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=5, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668420] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=6, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668424] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=7, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668429] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=8, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668433] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=9, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668437] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=10, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668442] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=11, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668446] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=12, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668451] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=13, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668454] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO SENDER STATUS](send_index=14, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668458] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO SENDER STATUS](send_index=15, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.668463] INFO  [COMMON] print_sender_status (ob_io_struct.cpp:716) [3139][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO SENDER STATUS](send_index=16, req_count=0, reservation_ts=9223372036854775807, group_limitation_ts=9223372036854775807, tenant_limitation_ts=9223372036854775807, proportion_ts=9223372036854775807)
[2023-07-23 11:04:40.684132] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196775}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480572433, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.684167] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=34] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196775}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480572433, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.684192] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=23] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480648217, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196775}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480572433, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.684231] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480683993 2023-07-23 03:04:40.683993|[process_begin] u=0 in_queue_time:7408753, receive_ts:1690081473275239, enqueue_ts:1690081480683979|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=56 |[check_priv] u=8 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=1 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=2 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=9 |[sync_auto_value] u=2 |[close_plan_begin] u=20 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=88 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=10 |[process_end] u=0 run_ts:1690081480683996|total_timeu=237)
[2023-07-23 11:04:40.684483] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079328-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.684497] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079328-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.684514] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.684592] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480683958 2023-07-23 03:04:40.683958|[process_begin] u=0 in_queue_time:4660131, receive_ts:1690081476023826, enqueue_ts:1690081480683949|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=3 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=32 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=3 |[plan_id] u=1 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=17 |[calc_partition_location_begin] u=445 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=0 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=6 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=39 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=21 |[close_das_task_begin] u=0 |[close_das_task_end] u=22 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=8 |[process_end] u=1 run_ts:1690081480683961|total_timeu=633)
[2023-07-23 11:04:40.684853] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079328-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.685193] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079328-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.685547] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079328-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.687396] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=15][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.687407] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081480631471], now0=[mts=1690081480631471], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.687415] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908686}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480630333, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.687439] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=21][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:40.687448] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908686}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480630333, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.687464] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=15] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480558737, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908686}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480630333, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.687492] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=27][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.687499] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.687503] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.687506] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.687509] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:40.687516] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=96, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:40.687523] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6] will sleep(sleep_us=96000, remain_us=19989014, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=96, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:40.692869] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.692887] INFO  [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:808) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1002, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=false, can_update_version=false, cur_leader_epoch=0, start_service_tstamp_=0, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0)
[2023-07-23 11:04:40.692908] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1}, delta=15177137, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.692918] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.692935] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1001}, delta=15285584, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.692948] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.692961] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1002}, delta=15614871, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.692969] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.692965] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.692983] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1003}, delta=15549012, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.692989] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.693005] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15285822, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.693021] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.704201] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.704244] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480704121 2023-07-23 03:04:40.704121|[process_begin] u=0 in_queue_time:5479430, receive_ts:1690081475224690, enqueue_ts:1690081480704107|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=28 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=2 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=2 |[parse_job_end] u=4 |[remote_task_completed] u=45 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29261}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=11 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=2 |[close_plan_begin] u=1 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=10 |[process_end] u=1 run_ts:1690081480704123|total_timeu=122)
[2023-07-23 11:04:40.713195] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3360][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713333] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3361][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713428] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3364][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713548] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3365][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713594] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3359][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713781] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3358][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=13] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713924] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3362][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713966] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3363][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.713972] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3357][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.714309] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=33][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.714321] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.714361] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480714211 2023-07-23 03:04:40.714211|[process_begin] u=0 in_queue_time:217335, receive_ts:1690081480496875, enqueue_ts:1690081480714202|[start_sql] u=0 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=24 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=3 |[pc_choose_plan] u=2 |[check_priv] u=2 |[plan_id] u=1 plan_id:18472|[do_open_plan_begin] u=1 plan_id:18472|[sql_start_stmt_begin] u=3 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=9 |[parse_job_end] u=7 |[remote_task_completed] u=61 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29263}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=10 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=2 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=9 |[process_end] u=1 run_ts:1690081480714213|total_timeu=149)
[2023-07-23 11:04:40.720027] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3371][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=5] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720502] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3369][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=10] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720512] INFO  [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:595) [3376][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=6] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720550] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3366][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720657] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3352][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=80/105, request doing=0/0)
[2023-07-23 11:04:40.720696] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3374][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=6] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720700] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3355][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720742] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3372][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720743] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3370][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720900] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3368][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=8] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.720955] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3351][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=73/90, request doing=0/0)
[2023-07-23 11:04:40.720965] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3345][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=87/100, request doing=0/0)
[2023-07-23 11:04:40.721074] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3346][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=75/102, request doing=0/0)
[2023-07-23 11:04:40.721086] INFO  [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:657) [3367][MysqlIO][T0][Y0-0000000000000000-0-0] [lt=7] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2023-07-23 11:04:40.721116] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3344][BatchIO][T0][Y0-0000000000000000-0-0] [lt=5] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=87/104, request doing=0/0)
[2023-07-23 11:04:40.721185] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3354][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=66/85, request doing=0/0)
[2023-07-23 11:04:40.721246] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3348][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=67/82, request doing=0/0)
[2023-07-23 11:04:40.721299] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3343][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=80/96, request doing=0/0)
[2023-07-23 11:04:40.721344] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3341][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=68/69, request doing=0/0)
[2023-07-23 11:04:40.721365] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3349][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=86/115, request doing=0/0)
[2023-07-23 11:04:40.721467] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3353][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=89/110, request doing=0/0)
[2023-07-23 11:04:40.721565] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3342][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=79/92, request doing=0/0)
[2023-07-23 11:04:40.721596] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3340][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=63/115, request doing=0/0)
[2023-07-23 11:04:40.721598] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3347][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=61/93, request doing=0/0)
[2023-07-23 11:04:40.721617] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3350][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=68/114, request doing=0/0)
[2023-07-23 11:04:40.721786] INFO  [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:633) [3339][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=72/111, request doing=0/0)
[2023-07-23 11:04:40.727344] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [3727][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=17] sql audit evict task end(evict_high_mem_level=412316860, evict_high_size_level=9000000, evict_batch_count=0, elapse_time=0, size_used=88203, mem_used=235011072)
[2023-07-23 11:04:40.730808] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.734549] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=0] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196776}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480622955, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.734582] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=33] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196776}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480622955, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.734611] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=28] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480648219, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196776}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480622955, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.734650] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081480734423 2023-07-23 03:04:40.734423|[process_begin] u=0 in_queue_time:8074288, receive_ts:1690081472660135, enqueue_ts:1690081480734409|[start_sql] u=1 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=50 |[check_priv] u=8 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=6 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=8 |[sync_auto_value] u=3 |[close_plan_begin] u=15 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=90 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=10 |[process_end] u=1 run_ts:1690081480734426|total_timeu=226)
[2023-07-23 11:04:40.735370] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FA-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.736209] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FA-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.737028] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FA-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.737080] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B0A-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.737095] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83272][T1001_Occam][T1][YB420A01819C-0005FDE72F6A9B0A-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.737842] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FA-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.739165] INFO  [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:63) [83125][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=29] report RowHolderMapper summary info(count=0, bkt_cnt=252)
[2023-07-23 11:04:40.740484] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:131, local_retry_times:131, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.740511] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=23][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.740524] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=13][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.740536] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.740557] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.740649] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=16][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.740672] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.749665] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480749655]})
[2023-07-23 11:04:40.749681] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480749676]})
[2023-07-23 11:04:40.749693] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.749705] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=11][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.749719] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.749726] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.749823] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480749816]})
[2023-07-23 11:04:40.749832] INFO  [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] gts statistics(tenant_id=1002, gts_rpc_cnt=0, get_gts_cache_cnt=51, get_gts_with_stc_cnt=2880, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=0, try_wait_gts_elapse_cnt=0)
[2023-07-23 11:04:40.749902] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.749913] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=11][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.749928] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=9][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.749952] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=24][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.749975] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=22] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.749998] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=20] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.750009] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B066-0-0] [lt=8] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480750006, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.750118] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1FE-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.750132] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1FE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.750288] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.754639] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=35] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480754632})
[2023-07-23 11:04:40.754643] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC40-0-0] [lt=1] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.755175] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.755269] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480754575 2023-07-23 03:04:40.754575|[process_begin] u=0 in_queue_time:4801254, receive_ts:1690081475953320, enqueue_ts:1690081480754566|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=44 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=0 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=27 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=5 |[do_open_plan_end] u=0 |[get_das_id_begin] u=29 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=13 |[calc_partition_location_begin] u=430 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=6 plan_id:427|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=29 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=21 |[close_das_task_begin] u=1 |[close_das_task_end] u=22 |[start_end_stmt] u=14 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=13 |[process_end] u=1 run_ts:1690081480754577|total_timeu=693)
[2023-07-23 11:04:40.757467] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=11][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.757479] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.757503] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15295344, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.757514] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.760546] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:167) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=8] sql audit evict mem start(evict_low_mem_level=28923920, evict_high_mem_level=49895441, size_used=40, mem_used=60312576)
[2023-07-23 11:04:40.760566] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:173) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=12] release old cannot free more memory
[2023-07-23 11:04:40.760573] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [83375][T1001_ReqMemEvi][T1001][Y0-0000000000000000-0-0] [lt=7] sql audit evict task end(evict_high_mem_level=49895441, evict_high_size_level=90000, evict_batch_count=1, elapse_time=28, size_used=0, mem_used=60312576)
[2023-07-23 11:04:40.763189] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1956-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.768467] WDIAG [SHARE] refresh (ob_task_define.cpp:382) [3138][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=13][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4038, dropped:370, tid:80219}, {errcode:-4122, dropped:5358, tid:45604}])
[2023-07-23 11:04:40.774431] INFO  [PALF] get_disk_usage (palf_env_impl.cpp:781) [3128][ServerGTimer][T1002][YB420A01819C-0005FDE72B4C6664-0-0] [lt=7] get_disk_usage(ret=0, capacity(MB):=60825, used(MB):=48597)
[2023-07-23 11:04:40.777428] INFO  [SERVER] runTimerTask (ob_eliminate_task.cpp:202) [83138][T1002_ReqMemEvi][T1002][Y0-0000000000000000-0-0] [lt=13] sql audit evict task end(evict_high_mem_level=510242114, evict_high_size_level=9000000, evict_batch_count=0, elapse_time=0, size_used=124373, mem_used=415948800)
[2023-07-23 11:04:40.783589] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:96, local_retry_times:96, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.783620] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=21][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.783631] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.783640] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.783662] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.783743] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.783755] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.783765] WDIAG [STORAGE.TRANS] post (ob_gts_rpc.cpp:248) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", server="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081480783737], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:40.783783] WDIAG [STORAGE.TRANS] query_gts_ (ob_gts_source.cpp:562) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=17][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", leader="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081480783737], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:40.783796] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.785382] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=54][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.785395] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=13][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.785401] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1, svr="10.1.129.154:1001", timeout=9999966, timeout_timestamp=1690081490785341)
[2023-07-23 11:04:40.785415] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=13] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.785425] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:40.785433] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:40.785437] WDIAG [SQL] open (ob_result_set.cpp:157) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:40.785444] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:40.785449] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"})
[2023-07-23 11:04:40.785458] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] execute failed(ret=-4122, tenant_id=1, executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:40.785466] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:40.785479] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.785485] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.785488] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.785537] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.785547] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:40.785553] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1, sql=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:40.785560] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1)
[2023-07-23 11:04:40.785566] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1)
[2023-07-23 11:04:40.785573] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] query failed(ret=-4122, conn=0x7fd4cac18050, start=1690081480785333, sql=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:40.785581] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=8][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:40.785586] WDIAG [SERVER] next_tenant_ (ob_iterate_private_virtual_table.cpp:255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=3][errcode=-4122] execute sql failed(ret=-4122, ret="OB_RPC_POST_ERROR", exec_tenant_id=1, cur_tenant_id=1, sql_=SELECT /*+ query_timeout(10000000) */ `tenant_id`, `frozen_scn`, `last_merged_scn`, `merge_start_time`, `is_merge_error`, `suspend_merging` FROM `__all_merge_info` WHERE 1=1 AND tenant_id = 1)
[2023-07-23 11:04:40.785594] WDIAG [SERVER] do_open (ob_iterate_private_virtual_table.cpp:138) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] switch to next tenant failed(ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:40.785599] WDIAG [SERVER] inner_get_next_row (ob_iterate_private_virtual_table.cpp:271) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] do open failed(ret=-4122, ret="OB_RPC_POST_ERROR")
[2023-07-23 11:04:40.785606] WDIAG [COMMON] get_next_row (ob_virtual_table_iterator.cpp:372) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] fail to inner get next row(ret=-4122, scan_param_={tablet_id:{id:1}, ls_id:{id:1}, column_ids:[16, 20, 23, 27, 22, 26], index_id:12291, key_ranges:[{"range":"table_id:12291,group_idx:0,(MIN;MAX)"}], ss_key_ranges:[{"range":"table_id:12291,group_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1690081485923826, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:1, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup:0, reserved:0}, sql_mode:12582912, reserved_cell_count:6, schema_version:1686281724066464, query_begin_schema_version:1686562084609584, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[{type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:0, res_buf_off:416, dyn_buf_header_offset:400, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484060}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT UNSIGNED", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:16, res_buf_off:424, dyn_buf_header_offset:408, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484110}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT UNSIGNED", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:32, res_buf_off:432, dyn_buf_header_offset:416, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd5154841c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:48, res_buf_off:440, dyn_buf_header_offset:424, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484270}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:64, res_buf_off:448, dyn_buf_header_offset:432, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd515484320}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:2, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:80, res_buf_off:456, dyn_buf_header_offset:440, res_buf_len:8, eval_flags_off:0, pvt_skip_off:0, expr_ctx_id:4294967295, extra:0, batch_idx_mask:0, this:0x7fd5154843d0}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7fd23e406ce8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, table_param:{table_id:12291, output_projector:[0, 1, 2, 3, 4, 5], aggregate_projector:[], output_sel_mask:[true, true, true, true, true, true], pad_col_projector:[], main_read_info:{schema_column_count:13, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:13, is_oracle_mode:false, cols_index:[0, 6, 9, 13, 8, 12], memtable_cols_index:[0, 4, 7, 11, 6, 10], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:8, allocator:0x7fcfd79c2530, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:23, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:27, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:26, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fcfd79c2530, is_inited:true}, cols_param:null, index_read_info:NULL}}, use_lob_locator:false, rowid_version:0, rowid_projector:[], enable_lob_locator_v2:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0}, need_scn:false, need_switch_param:false})
[2023-07-23 11:04:40.785779] WDIAG [COMMON] get_next_row (ob_virtual_table_iterator.cpp:472) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=172][errcode=-4122] get next row failed(ret=-4122)
[2023-07-23 11:04:40.785784] WDIAG [SQL.ENG] get_next_row_with_das (ob_table_scan_op.cpp:1683) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] get next row from das result failed(ret=-4122)
[2023-07-23 11:04:40.785788] WDIAG [SQL.ENG] inner_get_next_row_for_tsc (ob_table_scan_op.cpp:1870) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] fail to get next row from ObNewRowIterator(ret=-4122)
[2023-07-23 11:04:40.785793] WDIAG [SQL.ENG] inner_get_next_row_implement (ob_table_scan_op.cpp:1847) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] failed to get next row(ret=-4122)
[2023-07-23 11:04:40.785798] WDIAG [SQL.ENG] inner_get_next_row (ob_table_scan_op.cpp:2827) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] get next row failed(ret=-4122)
[2023-07-23 11:04:40.785802] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=3][errcode=-4122] inner get next row failed(ret=-4122, type=3, op="PHY_TABLE_SCAN", op_id=0)
[2023-07-23 11:04:40.785810] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] get next row from operator failed(ret=-4122)
[2023-07-23 11:04:40.785813] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] get next row from exec result failed(ret=-4122)
[2023-07-23 11:04:40.785819] WDIAG [SERVER] response_query_result (ob_query_driver.cpp:233) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=5][errcode=-4122] fail to iterate and response(ret=-4122, row_num=0, can_retry=true)
[2023-07-23 11:04:40.785827] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:101) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=7][errcode=-4122] response query result fail(ret=-4122)
[2023-07-23 11:04:40.785832] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:50, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:2, client_ret:-4122}, need_retry=true)
[2023-07-23 11:04:40.785841] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:114) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=9][errcode=-4122] result response failed, check if need retry(ret=-4122, cli_ret=-4122, retry_ctrl_.need_retry()=1)
[2023-07-23 11:04:40.785873] WDIAG [SERVER] response_result (ob_sync_plan_driver.cpp:121) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=4][errcode=-4122] close result set fail(cret=-4122)
[2023-07-23 11:04:40.785879] WDIAG [SERVER] do_process (obmp_query.cpp:846) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] execute query fail(ret=-4122, timeout_timestamp=1690081486023826)
[2023-07-23 11:04:40.785885] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6][errcode=-4122] try to execute again(ret=-4122, type=1, retry_type=2, timeout_remain=5237941)
[2023-07-23 11:04:40.785899] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081480784848 2023-07-23 03:04:40.784848|[process_begin] u=0 in_queue_time:4761021, receive_ts:1690081476023826, enqueue_ts:1690081480784839|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=30 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=15 |[calc_partition_location_begin] u=439 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=0 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=55 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=8 |[do_open_plan_end] u=5 |[get_location_cache_begin] u=6 |[close_plan_begin] u=33 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=368 |[close_das_task_begin] u=1 |[close_das_task_end] u=23 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=25 |[process_end] u=1 run_ts:1690081480784850|total_timeu=1050)
[2023-07-23 11:04:40.792919] WDIAG [PALF] convert_to_ts (scn.cpp:265) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4016] invalid scn should not convert to ts (val_=18446744073709551615)
[2023-07-23 11:04:40.792932] INFO  [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:527) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1002, server_version={version:{val:1690081465077463222}, total_part_count:4, valid_inner_part_count:1, valid_user_part_count:3}, server_version_delta=15715453, in_cluster_service=false, cluster_version={val:18446744073709551615}, min_cluster_version={val:18446744073709551615}, max_cluster_version={val:18446744073709551615}, get_cluster_version_err=0, cluster_version_delta=1690081480792916, cluster_service_master="10.1.129.154:1001", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=33997620, succ_cluster_heartbeat_count=33996921, cluster_heartbeat_interval=1000000, local_cluster_version={val:1690081407943046690}, local_cluster_delta=72849870, force_self_check=false, weak_read_refresh_interval=100000)
[2023-07-23 11:04:40.792971] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=32][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.792980] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.792991] WDIAG [STORAGE.TRANS] post_cluster_heartbeat_rpc_ (ob_tenant_weak_read_service.cpp:803) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] post cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", cluster_service_master="10.1.129.154:1001", tenant_id_=1002, req={req_server:"192.168.1.1:1001", version:{val:1690081465077463222}, valid_part_count:4, total_part_count:4, generate_timestamp:1690081480792963}, cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.793009] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.793014] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=6][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.793010] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.793024] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:759) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4122] post cluster heartbeat rpc fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1002, local_server_version={val:1690081465077463222}, valid_part_count=4, total_part_count=4, generate_timestamp=1690081480792963)
[2023-07-23 11:04:40.793023] WDIAG [STORAGE.TRANS] post_cluster_heartbeat_rpc_ (ob_tenant_weak_read_service.cpp:803) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] post cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", cluster_service_master="10.1.129.154:1001", tenant_id_=1, req={req_server:"192.168.1.1:1001", version:{val:1690081465406512168}, valid_part_count:1, total_part_count:1, generate_timestamp:1690081480793004}, cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.793046] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.793043] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:769) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=19][errcode=-4122] tenant weak read service do cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1002, last_post_cluster_heartbeat_tstamp_=1690081480692882, cluster_heartbeat_interval_=1000000, cluster_service_tablet_id={id:226}, cluster_service_master="10.1.129.154:1001")
[2023-07-23 11:04:40.793055] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:759) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] post cluster heartbeat rpc fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1, local_server_version={val:1690081465406512168}, valid_part_count=1, total_part_count=1, generate_timestamp=1690081480793004)
[2023-07-23 11:04:40.793064] WDIAG [STORAGE.TRANS] do_cluster_heartbeat_ (ob_tenant_weak_read_service.cpp:769) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] tenant weak read service do cluster heartbeat fail(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id_=1, last_post_cluster_heartbeat_tstamp_=1690081480692988, cluster_heartbeat_interval_=1000000, cluster_service_tablet_id={id:226}, cluster_service_master="10.1.129.154:1001")
[2023-07-23 11:04:40.793070] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1}, delta=15277212, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.793080] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1}, delta=15385897, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.793087] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.793085] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.793108] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=15] get wrs ts(ls_id={id:1001}, delta=15385659, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.793121] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.793141] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1002}, delta=15714946, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.793153] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.793171] INFO  [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:730) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=13] get_max_applied_scn(scn={val:1690081465143322587}, this={ls_id_:{id:1003}, role_:1, proposal_id_:54, palf_committed_end_lsn_:{lsn:375365428166}, last_check_scn_:{val:1690081465143322588}, max_applied_cb_scn_:{val:1690081465143322587}})
[2023-07-23 11:04:40.793191] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1003}, delta=15649087, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.793204] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.795065] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:40.795077] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=12][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196777}, scn:1690081480749156}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196777}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480683954, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.795108] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=31][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196777}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480683954, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.795262] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=153][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:40.795269] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.795276] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:40.795284] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.795288] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:40.795302] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=14][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:40.795306] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:40.795313] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702504005, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196777}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480683954, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196777}, scn:1690081480749156}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:40.795712] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=399][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:40.795718] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:40.795727] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:40.795731] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:40.795735] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:40.795740] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:40.795747] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:40.795751] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:40.795756] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:40.795760] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:40.795768] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:40.795776] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:40.795796] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:71, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:40.795810] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196777}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480683954, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.795839] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196777}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480683954, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.795867] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=27] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480749156, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196777}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480683954, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.795896] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:40.795905] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:40.795909] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:40.795917] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=92479322)
[2023-07-23 11:04:40.795932] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081480794966 2023-07-23 03:04:40.794966|[process_begin] u=0 in_queue_time:7519726, receive_ts:1690081473275239, enqueue_ts:1690081480794955|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=47 |[check_priv] u=8 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=7 |[insert_start] u=1 |[get_das_id_begin] u=11 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=252 |[sync_auto_value] u=427 |[close_plan_begin] u=66 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=85 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=35 |[process_end] u=1 run_ts:1690081480794969|total_timeu=965)
[2023-07-23 11:04:40.796160] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=0][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.796170] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=10][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.796175] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999653, timeout_timestamp=1690081481795806)
[2023-07-23 11:04:40.796184] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=8] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.796190] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:40.796194] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:40.796197] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:40.796200] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:40.796204] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"})
[2023-07-23 11:04:40.796209] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:40.796215] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=6][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:40.796223] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=6][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.796227] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.796230] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.796235] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.796241] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:40.796246] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:40.796249] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:40.796253] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:40.796257] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] query failed(ret=-4122, conn=0x7fd3ed9ce050, start=1690081480796114, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:40.796261] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:40.796265] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 200282)
[2023-07-23 11:04:40.796285] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:200282})
[2023-07-23 11:04:40.796299] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=13][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:200282}, add_timestamp:1690081480795783})
[2023-07-23 11:04:40.796628] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=6][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.796636] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=7][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.796642] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999680, timeout_timestamp=1690081481796304)
[2023-07-23 11:04:40.796648] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=6] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.796653] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:40.796657] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:40.796651] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=25][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.796660] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:40.796663] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:40.796661] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081480740640], now0=[mts=1690081480740640], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.796666] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"})
[2023-07-23 11:04:40.796671] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:40.796677] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=6][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:40.796688] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=10][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.796670] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:135, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.796691] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.796694] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.796693] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=21][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:40.796699] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.796704] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:40.796713] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=8][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:40.796703] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:135, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.796717] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:40.796722] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:40.796726] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] query failed(ret=-4122, conn=0x7fd508bf2050, start=1690081480796594, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:40.796730] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:40.796733] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847555)
[2023-07-23 11:04:40.796750] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847555})
[2023-07-23 11:04:40.796722] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=18] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480648221, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:135, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.796758] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=8][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847555}, add_timestamp:1690081480795787})
[2023-07-23 11:04:40.796758] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=35][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.796766] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.796771] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.796775] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.796779] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:40.796789] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=132, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:40.796796] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7] will sleep(sleep_us=100000, remain_us=14373052, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=132, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:40.797079] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.797087] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=8][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.797091] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999688, timeout_timestamp=1690081481796763)
[2023-07-23 11:04:40.797098] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=6] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.797106] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=7][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:40.797110] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:40.797113] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:40.797116] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:40.797119] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"})
[2023-07-23 11:04:40.797124] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:40.797129] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:40.797139] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=7][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.797143] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.797146] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.797151] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.797156] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:40.797160] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:40.797164] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:40.797167] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:40.797171] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] query failed(ret=-4122, conn=0x7fd3ccd24050, start=1690081480797047, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:40.797175] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:40.797178] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847591)
[2023-07-23 11:04:40.797194] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847591})
[2023-07-23 11:04:40.797202] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=8][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847591}, add_timestamp:1690081480795789})
[2023-07-23 11:04:40.797531] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.797538] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=8][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.797543] WDIAG [SQL.EXE] task_execute_v2 (ob_executor_rpc_impl.cpp:136) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] rpc task_execute fail(ret=-4122, tenant_id=1002, svr="10.1.129.154:1001", timeout=999680, timeout_timestamp=1690081481797206)
[2023-07-23 11:04:40.797549] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=6] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.797554] WDIAG [SQL.EXE] schedule (ob_remote_scheduler.cpp:53) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] execute with sql failed(ret=-4122)
[2023-07-23 11:04:40.797557] WDIAG [SQL] do_open_plan (ob_result_set.cpp:506) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail execute plan(ret=-4122)
[2023-07-23 11:04:40.797560] WDIAG [SQL] open (ob_result_set.cpp:157) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] execute plan failed(ret=-4122)
[2023-07-23 11:04:40.797563] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] open result set failed(ret=-4122)
[2023-07-23 11:04:40.797566] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] result set open failed(ret=-4122, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"})
[2023-07-23 11:04:40.797571] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] execute failed(ret=-4122, tenant_id=1002, executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, retry_cnt=0, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1689916332790784)
[2023-07-23 11:04:40.797580] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=10][errcode=-4122] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4122, err_:"OB_RPC_POST_ERROR", retry_type:0, client_ret:-4122}, need_retry=false)
[2023-07-23 11:04:40.797587] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.797590] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] result set close failed(ret=-4122)
[2023-07-23 11:04:40.797593] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] failed to close result(close_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.797598] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE713079329-0-0] [lt=2][errcode=-4122] failed to process record(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, record_ret=-4122, ret=-4122)
[2023-07-23 11:04:40.797603] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:833) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] failed to process final(executor={ObIExecutor:, sql:"SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592"}, aret=-4122, ret=-4122)
[2023-07-23 11:04:40.797607] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:2020) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] execute sql failed(ret=-4122, tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:40.797611] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:890) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] retry_while_no_tenant_resource failed(ret=-4122, tenant_id=1002)
[2023-07-23 11:04:40.797614] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1961) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] execute_read failed(ret=-4122, cluster_id=3, tenant_id=1002)
[2023-07-23 11:04:40.797618] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:125) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] query failed(ret=-4122, conn=0x7fd51fbf8050, start=1690081480797498, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:40.797622] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:63) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] read failed(ret=-4122)
[2023-07-23 11:04:40.797627] WDIAG [SHARE.LOCATION] inner_get_by_sql_ (ob_tablet_ls_service.cpp:395) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=4][errcode=-4122] fail to execute sql(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, sql=SELECT ls_id, ORA_ROWSCN from __all_tablet_to_ls WHERE tablet_id = 1152921504606847592)
[2023-07-23 11:04:40.797643] WDIAG [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:340) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail to get log stream info(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1002, tablet_id={id:1152921504606847592})
[2023-07-23 11:04:40.797650] WDIAG [SHARE.LOCATION] batch_process_tasks (ob_tablet_ls_service.cpp:222) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=6][errcode=-4122] fail to renew tablet_cache(ret=-4122, ret="OB_RPC_POST_ERROR", task={tenant_id:1002, tablet_id:{id:1152921504606847592}, add_timestamp:1690081480795793})
[2023-07-23 11:04:40.797655] WDIAG [SERVER] batch_process_tasks (ob_uniq_task_queue.h:498) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=5][errcode=-4122] fail to batch process task(ret=-4122)
[2023-07-23 11:04:40.797658] WDIAG [SERVER] run1 (ob_uniq_task_queue.h:449) [3288][TbltLSSrv1][T0][YB420A01819C-0005FDE713079329-0-0] [lt=3][errcode=-4122] fail to batch execute task(ret=-4122, tasks.count()=4)
[2023-07-23 11:04:40.805071] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=40][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006})
[2023-07-23 11:04:40.805091] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=21][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.805108] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=17][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.805114] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.805122] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:40.805145] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=23][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:40.805154] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:40.805158] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:40.805163] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:40.806610] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:40.806656] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=20][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006})
[2023-07-23 11:04:40.806679] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=23][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.806697] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=18][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.806703] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.806708] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.806716] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:40.806720] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:40.806725] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.806730] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.806734] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.806739] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:40.806748] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:40.806752] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:40.806756] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:40.806762] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=3][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:40.806769] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=6][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:40.806774] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:40.806778] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] fail to handle text query(stmt=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882, ret=-4722)
[2023-07-23 11:04:40.806788] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:56, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:40.806798] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=9][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=select /*+ MONITOR_AGENT READ_CONSISTENCY(WEAK) */ status, (case when STOP_TIME is NULL then 0 else TIMESTAMPDIFF(SECOND, STOP_TIME, CURRENT_TIMESTAMP) end) as stopped_duration_seconds from DBA_OB_SERVERS where svr_ip = '192.168.1.1' and svr_port = 2882)
[2023-07-23 11:04:40.806805] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=4][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=4417886)
[2023-07-23 11:04:40.806826] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=5][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949678530, cache_obj->added_lc()=false, cache_obj->get_object_id()=20505, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.806847] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=10] [packet retry query](TRACE=begin_ts=1690081480805030 2023-07-23 03:04:40.805030|[process_begin] u=0 in_queue_time:5580340, receive_ts:1690081475224690, enqueue_ts:1690081480805021|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=32 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[pc_choose_plan] u=94 |[transform_with_outline_begin] u=54 |[transform_with_outline_end] u=1 |[resolve_begin] u=2 |[resolve_end] u=488 |[transform_begin] u=19 |[optimize_begin] u=755 |[tl_calc_part_id_end] u=119 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=188 |[query_end] u=57 |[process_end] u=1 run_ts:1690081480805033|total_timeu=1816)
[2023-07-23 11:04:40.815163] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=22][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006})
[2023-07-23 11:04:40.815184] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=21][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551605, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.815202] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=18][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=5)
[2023-07-23 11:04:40.815208] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.815214] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551605, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551607, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551609, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551613, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551611, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551605, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551607, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551609, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551613, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}, {table_location_key:18446744073709551611, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:40.815298] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=76][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551605, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551607, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551609, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551613, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}, {loc_meta:{table_loc_id:18446744073709551611, ref_table_id:116, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:40.815325] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=27][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:40.815330] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:40.815336] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:40.815855] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.815872] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=13][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.815886] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.815899] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=9][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.815911] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:40.819663] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=0][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.819680] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.819689] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:40.819698] WDIAG [SQL.ENG] refresh_statistics (ob_px_tenant_target_monitor.cpp:114) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122] query statistics failed(ret=-4122)
[2023-07-23 11:04:40.819710] WDIAG [COMMON] operator() (ob_px_target_mgr.h:143) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4122] target monitor refresh statistics failed(ret=-4122, px_tenant_info={v:1}, px_res_info->get_target_monitor()={is_init:true, tenant_id:1, server:"192.168.1.1:1001", dummy_cache_leader:"10.1.129.154:1001", role:2})
[2023-07-23 11:04:40.819737] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=22][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.819747] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:398) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.819757] WDIAG [SQL.ENG] query_statistics (ob_px_tenant_target_monitor.cpp:272) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4122] send rpc to query statistics failed, need send refresh all(ret=-4122)
[2023-07-23 11:04:40.819744] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7] total rowcount, use statistics(table_meta_info_.table_row_count_=24, table_meta_info_.average_row_size_=1.170000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=2.808000000000000000e+03)
[2023-07-23 11:04:40.820669] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=24][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006})
[2023-07-23 11:04:40.820689] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=20][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:116, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:116, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480750006}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:40.820707] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=17][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.820716] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=8][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:40.820722] WDIAG [SQL.OPT] select_location (ob_log_plan.cpp:2014) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] fail to select replicas(ret=-4722, tbl_loc_list.count()=1, optimizer_context_.get_local_server_addr()="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:40.820728] WDIAG [SQL.JO] generate_base_table_paths (ob_join_order.cpp:6258) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] failed to select location(ret=-4722)
[2023-07-23 11:04:40.820736] WDIAG [SQL.JO] generate_normal_base_table_paths (ob_join_order.cpp:6229) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=8][errcode=-4722] failed to generate access paths(ret=-4722)
[2023-07-23 11:04:40.820741] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807)
[2023-07-23 11:04:40.820747] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.820754] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.820760] WDIAG [SQL.JO] generate_subquery_paths (ob_join_order.cpp:6498) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] failed to optimize sub-select(ret=-4722)
[2023-07-23 11:04:40.820765] WDIAG [SQL.JO] generate_normal_subquery_paths (ob_join_order.cpp:6466) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] failed to generate subquery path(ret=-4722)
[2023-07-23 11:04:40.820771] WDIAG [SQL.OPT] generate_plan_tree (ob_log_plan.cpp:7345) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] failed to generate the access path for the single-table query(ret=-4722, get_optimizer_context().get_query_ctx()->get_sql_stmt()=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807)
[2023-07-23 11:04:40.820777] WDIAG [SQL.OPT] generate_raw_plan_for_plain_select (ob_select_log_plan.cpp:4041) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] failed to generate plan tree for plain select(ret=-4722)
[2023-07-23 11:04:40.820781] WDIAG [SQL.OPT] generate_raw_plan (ob_log_plan.cpp:11201) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=3][errcode=-4722] fail to generate normal raw plan(ret=-4722)
[2023-07-23 11:04:40.820786] WDIAG [SQL.OPT] generate_plan (ob_log_plan.cpp:11165) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] fail to generate raw plan(ret=-4722)
[2023-07-23 11:04:40.820790] WDIAG [SQL.OPT] optimize (ob_optimizer.cpp:60) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] failed to perform optimization(ret=-4722)
[2023-07-23 11:04:40.820796] WDIAG [SQL] optimize_stmt (ob_sql.cpp:3255) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] Failed to optimize logical plan(ret=-4722)
[2023-07-23 11:04:40.820806] WDIAG [SQL] generate_plan (ob_sql.cpp:3001) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=9][errcode=-4722] Failed to optimizer stmt(ret=-4722)
[2023-07-23 11:04:40.820812] WDIAG [SQL] generate_physical_plan (ob_sql.cpp:2818) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=3][errcode=-4722] failed to generate plan(ret=-4722)
[2023-07-23 11:04:40.820819] WDIAG [SQL] handle_physical_plan (ob_sql.cpp:4456) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] Failed to generate plan(ret=-4722, result.get_exec_context().need_disconnect()=false)
[2023-07-23 11:04:40.820825] WDIAG [SQL] handle_text_query (ob_sql.cpp:2385) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=5][errcode=-4722] fail to handle physical plan(ret=-4722)
[2023-07-23 11:04:40.820829] WDIAG [SQL] stmt_query (ob_sql.cpp:206) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=3][errcode=-4722] fail to handle text query(stmt=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807, ret=-4722)
[2023-07-23 11:04:40.820836] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=4][errcode=-4722] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:4, local_retry_times:0, err_:-4722, err_:"OB_LS_LOCATION_LEADER_NOT_EXIST", retry_type:2, client_ret:-4722}, need_retry=true)
[2023-07-23 11:04:40.820845] WDIAG [SERVER] do_process (obmp_query.cpp:793) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=9][errcode=-4722] run stmt_query failed, check if need retry(ret=-4722, cli_ret=-4722, retry_ctrl_.need_retry()=1, sql=SELECT /*+READ_CONSISTENCY(WEAK)*/ zone, status, 0 AS is_merging, region, idc AS spare4, type AS spare5 FROM oceanbase.DBA_OB_ZONES LIMIT 9223372036854775807)
[2023-07-23 11:04:40.820853] WDIAG [SERVER] do_process (obmp_query.cpp:894) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=-4722] try to execute again(ret=-4722, type=1, retry_type=2, timeout_remain=9676022)
[2023-07-23 11:04:40.820881] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=6][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949692586, cache_obj->added_lc()=false, cache_obj->get_object_id()=20506, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:40.820902] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=9] [packet retry query](TRACE=begin_ts=1690081480815122 2023-07-23 03:04:40.815122|[process_begin] u=0 in_queue_time:318247, receive_ts:1690081480496875, enqueue_ts:1690081480815112|[start_sql] u=1 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=28 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[pc_choose_plan] u=174 |[transform_with_outline_begin] u=38 |[transform_with_outline_end] u=1 |[resolve_begin] u=3 |[resolve_end] u=646 |[transform_begin] u=20 |[optimize_begin] u=3446 |[tl_calc_part_id_end] u=246 |[get_location_cache_begin] u=0 |[storage_estimation_begin] u=68 |[storage_estimation_end] u=855 |[get_location_cache_begin] u=177 |[query_end] u=64 |[process_end] u=2 run_ts:1690081480815126|total_timeu=5779)
[2023-07-23 11:04:40.830795] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=6][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.836402] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [83154][T1002_HAService][T1002][Y0-0000000000000000-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}, {id:1002}, {id:1003}])
[2023-07-23 11:04:40.838323] INFO  [STORAGE] runTimerTask (ob_tenant_tablet_scheduler.cpp:147) [3742][T1_SSTableGC][T1][Y0-0000000000000000-0-0] [lt=17] SSTableGCTask(cost_ts=3986)
[2023-07-23 11:04:40.839666] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.839678] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081480783733], now0=[mts=1690081480783733], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.839689] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908687}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480782889, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.839716] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=24][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:40.839728] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908687}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480782889, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.839749] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=20] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480749155, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908687}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480782889, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.839784] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=35][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.839793] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=9][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.839800] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.839805] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.839811] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:40.839819] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=97, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:40.839828] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8] will sleep(sleep_us=97000, remain_us=19836709, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=97, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:40.845524] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=25][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:40.845535] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=11][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196778}, scn:1690081480749158}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196778}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480734404, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.845565] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=29][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196778}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480734404, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.845715] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=150][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:40.845722] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.845734] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:40.845741] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.845747] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:40.845752] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:40.845757] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:40.845762] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702504006, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196778}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480734404, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196778}, scn:1690081480749158}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:40.846145] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=382][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:40.846150] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:40.846159] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:40.846162] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:40.846166] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:40.846172] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:40.846180] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:40.846184] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:40.846189] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:40.846193] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:40.846198] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:40.846202] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:40.846220] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:77, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:40.846234] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196778}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480734404, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.846262] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196778}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480734404, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.846288] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=25] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480749158, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196778}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480734404, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.846322] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=29][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:40.846329] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:40.846335] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:40.846342] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=91813793)
[2023-07-23 11:04:40.846358] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7] [packet retry query](TRACE=begin_ts=1690081480845418 2023-07-23 03:04:40.845418|[process_begin] u=0 in_queue_time:8185282, receive_ts:1690081472660135, enqueue_ts:1690081480845407|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=52 |[check_priv] u=8 |[plan_id] u=1 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=15 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=242 |[sync_auto_value] u=410 |[close_plan_begin] u=57 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=87 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=35 |[process_end] u=1 run_ts:1690081480845420|total_timeu=939)
[2023-07-23 11:04:40.846973] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F5-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.846997] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.847784] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.848535] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.849278] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3287][TbltLSSrv0][T1002][YB420A01819C-0005FDE64EC793F5-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.850170] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480850158]})
[2023-07-23 11:04:40.850187] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480850182]})
[2023-07-23 11:04:40.850193] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=18][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.850195] INFO  [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8] gts statistics(tenant_id=1001, gts_rpc_cnt=0, get_gts_cache_cnt=50, get_gts_with_stc_cnt=3232, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=0, try_wait_gts_elapse_cnt=0)
[2023-07-23 11:04:40.850205] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.850212] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.850224] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.850230] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.850349] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480850341]})
[2023-07-23 11:04:40.850379] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.850389] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=9][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.850404] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=10][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.850428] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=23][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.850445] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=16] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.850464] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=17] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.850473] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B067-0-0] [lt=7] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480850471, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.851001] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C34-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.851193] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B1FF-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.855541] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=27] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480855535})
[2023-07-23 11:04:40.855546] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC41-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.855966] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.856044] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480855480 2023-07-23 03:04:40.855480|[process_begin] u=0 in_queue_time:4902159, receive_ts:1690081475953320, enqueue_ts:1690081480855472|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=3 |[session] u=4 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=38 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=3 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=22 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20478|[do_open_plan_begin] u=2 plan_id:20478|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=5 |[do_open_plan_end] u=0 |[get_das_id_begin] u=26 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=12 |[calc_partition_location_begin] u=333 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=0 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:427|[do_open_plan_begin] u=6 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=24 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=17 |[close_das_task_begin] u=1 |[close_das_task_end] u=17 |[start_end_stmt] u=13 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=11 |[process_end] u=0 run_ts:1690081480855484|total_timeu=563)
[2023-07-23 11:04:40.857524] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.857550] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15395326, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.857559] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.858377] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B200-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.859481] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=12] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73134578892, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:40.860645] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=17] Wash time detail, (compute_wash_size_time=82, refresh_score_time=671, wash_time=476)
[2023-07-23 11:04:40.863622] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=17] Cache replace map node details(ret=0, replace_node_count=0, replace_time=9458, replace_start_pos=22800000, replace_num=100000)
[2023-07-23 11:04:40.863707] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1957-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.869122] INFO  [PALF] runTimerTask (block_gc_timer_task.cpp:101) [83280][T1001_PalfGC][T1001][Y0-0000000000000000-0-0] [lt=26] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"192.168.1.1:1001", log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1001", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):6758, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95}, disk_opts_for_recycling_blocks:{log_disk_size(MB):6758, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95}, status:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}})
[2023-07-23 11:04:40.873100] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83283][T1001_LogUpdate][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.873891] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B201-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.879967] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=10] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=19312, clean_start_pos=9600000, clean_num=200000)
[2023-07-23 11:04:40.886150] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.886245] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480885740 2023-07-23 03:04:40.885740|[process_begin] u=0 in_queue_time:4861914, receive_ts:1690081476023826, enqueue_ts:1690081480885733|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=23 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=1 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=12 |[calc_partition_location_begin] u=331 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=0 |[plan_id] u=1 plan_id:17904|[do_open_plan_begin] u=5 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=25 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=3 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[close_plan_begin] u=50 |[close_das_task_begin] u=1 |[close_das_task_end] u=18 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=0 |[query_end] u=6 |[process_end] u=1 run_ts:1690081480885742|total_timeu=504)
[2023-07-23 11:04:40.892984] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=14] get wrs ts(ls_id={id:1}, delta=15377151, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.892997] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.893014] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1001}, delta=15485598, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.893022] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.893038] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=11] get wrs ts(ls_id={id:1002}, delta=15814885, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.893045] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.893057] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] get wrs ts(ls_id={id:1003}, delta=15749026, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.893065] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.893066] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] get wrs ts(ls_id={id:1}, delta=15485836, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.893075] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.896862] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:132, local_retry_times:132, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.896880] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=15][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.896886] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=7][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.896892] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=5][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.896909] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.896962] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=0][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.903180] INFO  [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:100) [83392][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=36] tx gc loop thread is running(MTL_ID()=1001)
[2023-07-23 11:04:40.903191] INFO  [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:107) [83392][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=11] try gc retain ctx
[2023-07-23 11:04:40.903912] INFO  [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:226) [83392][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=3] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1001, *ls_ptr={ls_meta:{tenant_id:1001, ls_id:{id:1}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690061231333212449}, clog_base_lsn:{lsn:99113742336}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:18446744073709551615}, tablet_change_checkpoint_scn:{val:0}, all_id_meta:{id_meta:[{limited_id:1690081473543177570, latest_log_ts:{val:1690081456904556569}}, {limited_id:150000001, latest_log_ts:{val:1690061231333212449}}, {limited_id:3720000001, latest_log_ts:{val:1690075632203527680}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4919f0030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:1, is_inited:true}})
[2023-07-23 11:04:40.905976] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.906007] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480905916 2023-07-23 03:04:40.905916|[process_begin] u=0 in_queue_time:5681226, receive_ts:1690081475224690, enqueue_ts:1690081480905908|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=21 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:17927|[do_open_plan_begin] u=1 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[parse_job_begin] u=1 |[parse_job_end] u=3 |[remote_task_completed] u=33 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29265}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=8 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=2 |[close_plan_begin] u=1 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=7 |[process_end] u=0 run_ts:1690081480905918|total_timeu=90)
[2023-07-23 11:04:40.906055] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:40.906066] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=11][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196779}, scn:1690081480850194}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196779}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480795582, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.906097] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=30][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196779}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480795582, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.906248] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=151][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:40.906254] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.906261] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:40.906270] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=10][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.906274] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:40.906280] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:40.906284] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:40.906298] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702504007, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196779}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480795582, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196779}, scn:1690081480850194}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:40.906695] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=397][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:40.906700] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:40.906708] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:40.906712] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:40.906716] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:40.906720] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:40.906727] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:40.906731] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:40.906738] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:40.906742] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:40.906747] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:40.906751] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:40.906783] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:72, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:40.906798] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196779}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480795582, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.906826] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196779}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480795582, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.906853] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480850194, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196779}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480795582, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.906881] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:40.906890] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:40.906896] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:40.906904] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=92368336)
[2023-07-23 11:04:40.906919] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081480905948 2023-07-23 03:04:40.905948|[process_begin] u=0 in_queue_time:7630708, receive_ts:1690081473275239, enqueue_ts:1690081480905938|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=51 |[check_priv] u=9 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=3 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=9 |[insert_start] u=0 |[get_das_id_begin] u=13 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=238 |[sync_auto_value] u=430 |[close_plan_begin] u=73 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=82 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=37 |[process_end] u=1 run_ts:1690081480905950|total_timeu=970)
[2023-07-23 11:04:40.907223] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FB-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.907238] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.907382] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307932A-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.907473] INFO  [PALF] log_loop_ (log_loop_thread.cpp:122) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=22] LogLoopThread round_cost_time(round_cost_time=2)
[2023-07-23 11:04:40.907715] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.908169] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FB-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.912892] INFO  [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:186) [3743][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=27] start do ls ha handler(ls_id_array_=[{id:1}])
[2023-07-23 11:04:40.923713] INFO  [PALF] log_loop_ (log_loop_thread.cpp:122) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=42] LogLoopThread round_cost_time(round_cost_time=2)
[2023-07-23 11:04:40.926163] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.926193] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480926099 2023-07-23 03:04:40.926099|[process_begin] u=0 in_queue_time:429223, receive_ts:1690081480496875, enqueue_ts:1690081480926091|[start_sql] u=0 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=18 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:18472|[do_open_plan_begin] u=1 plan_id:18472|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=3 |[parse_job_end] u=6 |[remote_task_completed] u=34 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29267}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=7 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=1 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=7 |[process_end] u=0 run_ts:1690081480926100|total_timeu=93)
[2023-07-23 11:04:40.930855] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:40.935550] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:260) [3737][T1_ArcSender][T1][YB420A01819C-0005FDE623B78DB5-0-0] [lt=13][errcode=-4018] try free send task failed(ret=-4018)
[2023-07-23 11:04:40.935564] INFO  [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:265) [3737][T1_ArcSender][T1][YB420A01819C-0005FDE623B78DB5-0-0] [lt=14] ObArchiveSender is running(thread_index=0)
[2023-07-23 11:04:40.936897] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:97, local_retry_times:97, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:40.936935] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=26][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.936948] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=13][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:40.936959] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.936980] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:40.937073] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=16][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:40.937086] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:40.937099] WDIAG [STORAGE.TRANS] post (ob_gts_rpc.cpp:248) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", server="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081480937067], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:40.937119] WDIAG [STORAGE.TRANS] query_gts_ (ob_gts_source.cpp:562) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=21][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", leader="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081480937067], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:40.937137] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=16][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:40.950686] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480950676]})
[2023-07-23 11:04:40.950698] INFO  [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:516) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=-4122, ret="OB_RPC_POST_ERROR", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081480950676]})
[2023-07-23 11:04:40.950706] INFO  [STORAGE.TRANS] operator() (ob_ts_mgr.h:171) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=7] refresh gts functor(ret=-4122, ret="OB_RPC_POST_ERROR", gts_tenant_info={v:1})
[2023-07-23 11:04:40.950718] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081480950713]})
[2023-07-23 11:04:40.950726] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.950743] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=15][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.950765] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.950777] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=11][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.950860] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081480950853]})
[2023-07-23 11:04:40.951022] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=15][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.951040] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=16][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.951064] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=18][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.951101] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=37][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.951134] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=31] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.951169] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=30] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.951183] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B068-0-0] [lt=11] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480951180, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.951448] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B202-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.951748] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C27-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.951774] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C27-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.952822] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.952844] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=22][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081480896955], now0=[mts=1690081480896955], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.952863] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=19][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:136, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.952912] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=46][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:40.952936] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=24][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:136, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.952976] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=39] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480850193, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:136, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.953047] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=70][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.953065] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=16][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.953076] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=11][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.953087] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.953098] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=10][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:40.953112] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=133, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:40.953129] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=15] will sleep(sleep_us=100000, remain_us=14216720, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=133, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:40.956374] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=29] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081480956369})
[2023-07-23 11:04:40.956379] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC42-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:40.956459] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:40.956469] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=10][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196780}, scn:1690081480950190}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196780}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480845975, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.956500] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=30][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196780}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480845975, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.956646] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=145][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:40.956652] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.956657] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:40.956665] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:40.956669] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:40.956674] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:40.956678] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:40.956811] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.956893] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480956326 2023-07-23 03:04:40.956326|[process_begin] u=0 in_queue_time:5003006, receive_ts:1690081475953320, enqueue_ts:1690081480956319|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=32 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=2 |[tl_calc_part_id_end] u=2 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=0 |[plc_serialize_end] u=1 |[tl_calc_part_id_end] u=22 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:20478|[do_open_plan_begin] u=1 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=4 |[do_open_plan_end] u=0 |[get_das_id_begin] u=33 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=10 |[calc_partition_location_begin] u=335 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:427|[do_open_plan_begin] u=5 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[remote_task_completed] u=31 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=1 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=0 |[close_plan_begin] u=3 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=16 |[close_das_task_begin] u=1 |[close_das_task_end] u=19 |[start_end_stmt] u=16 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=10 |[process_end] u=1 run_ts:1690081480956328|total_timeu=566)
[2023-07-23 11:04:40.956685] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702504008, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196780}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480845975, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196780}, scn:1690081480950190}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:40.957072] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=386][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:40.957077] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:40.957084] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:40.957088] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:40.957091] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:40.957095] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:40.957100] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:40.957106] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:40.957110] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:40.957115] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:40.957119] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:40.957124] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:40.957142] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:78, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:40.957156] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196780}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480845975, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.957184] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196780}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480845975, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:40.957208] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480950190, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196780}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480845975, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:40.957238] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=26][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:40.957245] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:40.957249] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:40.957256] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=91702879)
[2023-07-23 11:04:40.957271] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081480956358 2023-07-23 03:04:40.956358|[process_begin] u=0 in_queue_time:8296222, receive_ts:1690081472660135, enqueue_ts:1690081480956348|[start_sql] u=0 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=52 |[check_priv] u=8 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=2 |[value_start_calc_row] u=2 |[value_after_calc_row] u=7 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=230 |[sync_auto_value] u=410 |[close_plan_begin] u=55 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=81 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=32 |[process_end] u=1 run_ts:1690081480956360|total_timeu=912)
[2023-07-23 11:04:40.957583] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.957612] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15496379, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.957624] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.958103] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879365-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.958133] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879365-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.958555] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.958570] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.958578] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=7][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.958591] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.958597] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.958782] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.958796] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=12][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.958809] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=8][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.958831] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=22][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.958849] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=17] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.958870] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=19] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.959132] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879365-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.959360] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3716][T1_LSMetaCh][T1][YB420A01819C-0005FDE71F9AC3BE-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.960109] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879365-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.961068] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879365-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.964281] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1958-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.970617] WDIAG [PALF] try_fetch_log (log_sliding_window.cpp:1779) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=0] get_fetch_log_dst failed or invalid(ret=0, palf_id=1003, self="192.168.1.1:1001", fetch_log_dst="192.168.1.1:1001")
[2023-07-23 11:04:40.971173] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C35-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.971190] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C35-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.974449] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3633][T1_LogUpdater][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.974493] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.974506] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=12][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.974527] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.974536] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=9][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.974682] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=13][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.974697] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=14][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.974714] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=12][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.974746] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=31][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.974776] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=29] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.974809] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=30] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.974822] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B069-0-0] [lt=10] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480974820, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.987456] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.987610] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081480986606 2023-07-23 03:04:40.986606|[process_begin] u=0 in_queue_time:4962779, receive_ts:1690081476023826, enqueue_ts:1690081480986591|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=7 |[session] u=2 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=49 |[plc_serialize_begin] u=2 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=2 |[pc_choose_plan] u=1 |[check_priv] u=4 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=3 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_das_id_begin] u=4 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=2 |[do_local_das_task_end] u=23 |[calc_partition_location_begin] u=684 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=2 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=10 plan_id:17904|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=54 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=5 |[start_end_stmt] u=1 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=70 |[close_das_task_begin] u=1 |[close_das_task_end] u=34 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=12 |[process_end] u=1 run_ts:1690081480986615|total_timeu=1002)
[2023-07-23 11:04:40.990517] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.990670] INFO  [CLOG] handle_sync_request (log_request_handler.cpp:486) [83648][T1002_L0_G1][T1002][YB420A01819C-0005FDE7216A81B7-0-0] [lt=15] i am not leader(ret=-4038, palf_id=1003, req={get_type:1}, role=2, is_pending_state=true)
[2023-07-23 11:04:40.990729] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4038] execute rpc fail(ret=-4038, dst="192.168.1.1:1001")
[2023-07-23 11:04:40.990746] WDIAG log_user_error_and_warn (ob_rpc_proxy.cpp:252) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4038] 
[2023-07-23 11:04:40.990763] WDIAG [CLOG] post_sync_request_to_server_ (log_net_service.h:291) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4038] ObLogRpc post_sync_request failed(ret=-4038, palf_id=1003, req={get_type:1}, server="192.168.1.1:1001")
[2023-07-23 11:04:40.990779] WDIAG [CLOG] get_leader_max_scn_ (palf_handle_impl.cpp:4223) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=16][errcode=-4038] get_palf_max_scn failed(ret=-4038, palf_id=1003)
[2023-07-23 11:04:40.990792] WDIAG [CLOG] is_in_sync_ (palf_handle_impl.cpp:4182) [83043][T1002_LogUpdate][T1002][Y0-0000000000000000-0-0] [lt=11][errcode=-4038] get_palf_max_scn failed(ret=-4038, self="192.168.1.1:1001", palf_id=1003)
[2023-07-23 11:04:40.991029] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C42-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.991287] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3277][UserLocAsyncUp3][T1001][YB420A01819C-0005FDE628779C28-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.991322] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF6-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.991344] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3275][UserLocAsyncUp1][T1001][YB420A01819C-0005FDE627F79BF6-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.991580] INFO  [PALF] is_pending_log_clear_ (log_state_mgr.cpp:747) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=17] follower pending log not clear, need wait(bool_ret=false, palf_id=1003, last_slide_end_lsn={lsn:375365428166}, pending_end_lsn={lsn:375365428400})
[2023-07-23 11:04:40.993028] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.993039] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.993064] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=15476995, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.993075] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.993091] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1001}, delta=15585442, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.993100] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=8] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.993113] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] get wrs ts(ls_id={id:1002}, delta=15914729, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.993119] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:40.993121] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=6] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.993124] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:40.993128] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:40.993131] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=7] get wrs ts(ls_id={id:1003}, delta=15848870, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.993134] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081480937062], now0=[mts=1690081480937062], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:40.993137] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=5] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.993152] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15585680, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:40.993144] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908688}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480936531, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.993162] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:40.993168] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.993171] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=24][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:40.993176] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.993189] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:40.993183] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908688}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480936531, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:40.993198] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=8][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.993202] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=18] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081480850196, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908688}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480936531, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:40.993239] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=36][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:40.993248] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:40.993254] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:40.993258] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:40.993264] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=5][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:40.993272] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=98, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:40.993281] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8] will sleep(sleep_us=98000, remain_us=19683257, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=98, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:40.993380] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=11][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:40.993393] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=13][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:40.993409] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=11][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.993439] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=29][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.993464] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=25] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:40.993490] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=23] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:40.993501] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06A-0-0] [lt=8] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081480993499, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:40.994032] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3276][UserLocAsyncUp2][T1001][YB420A01819C-0005FDE629179C36-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:40.998501] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:260) [83148][T1002_ArcSender][T1002][YB420A01819C-0005FDE628378DB5-0-0] [lt=18][errcode=-4018] try free send task failed(ret=-4018)
[2023-07-23 11:04:40.999761] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:224) [83381][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=7] ====== [tabletgc] timer task ======(GC_CHECK_INTERVAL=5000000)
[2023-07-23 11:04:40.999773] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:254) [83381][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=9] [tabletgc] task check ls(ls->get_ls_id()={id:1}, is_tablet_gc=false)
[2023-07-23 11:04:40.999782] INFO  [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:282) [83381][T1001_ObTimer][T1001][Y0-0000000000000000-0-0] [lt=7] [tabletgc] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1)
[2023-07-23 11:04:41.003529] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:131) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] ====== checkpoint timer task ======
[2023-07-23 11:04:41.004330] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] succ to get rec scn(*this={this:0x7fd46b604030, ls_id:{id:1}, tenant_id:1002, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1690048802316336799}, last_push_gc_task_ts_:1690081221495637, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.004346] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=16] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc4080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1690048804114207}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc4080, snapshot_version:{val:1690048804114207}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.004364] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=15] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689738033637759567}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689738033637759567}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1})
[2023-07-23 11:04:41.004376] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd4f23e6250}, min_rec_scn={val:1690080742373882309}, ls_id_={id:1})
[2023-07-23 11:04:41.004828] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] get rec log scn(service_type_=0, rec_log_ts={val:1690081457643664628})
[2023-07-23 11:04:41.004835] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=1, rec_log_ts={val:1690081371645352093})
[2023-07-23 11:04:41.004842] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=4] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080742373882309}, checkpoint_scn_in_ls_meta={val:1690080742373882309}, ls_id={id:1}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:41.004855] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1153) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.1:1001", has_set_deleted:false}, curr_base_lsn={lsn:8388096000}, new_base_lsn={lsn:8388096000}, lsn={lsn:8388096000})
[2023-07-23 11:04:41.004866] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:184) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:8388096000}, ls->get_ls_id()={id:1})
[2023-07-23 11:04:41.005670] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] succ to get rec scn(*this={this:0x7fd46b686030, ls_id:{id:1001}, tenant_id:1002, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1689841617974500487}, last_push_gc_task_ts_:1689841618653371, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.005682] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc4d90, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1689841618655957}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc4d90, snapshot_version:{val:1689841618655957}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.005696] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=13] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689736139869551129}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689736139869551129}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1001})
[2023-07-23 11:04:41.005706] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd465388250}, min_rec_scn={val:1690080741701524066}, ls_id_={id:1001})
[2023-07-23 11:04:41.006034] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080741701524066}, checkpoint_scn_in_ls_meta={val:1690080741701524066}, ls_id={id:1001}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:41.006043] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1153) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"192.168.1.1:1001", has_set_deleted:false}, curr_base_lsn={lsn:703996121088}, new_base_lsn={lsn:703996121088}, lsn={lsn:703996121088})
[2023-07-23 11:04:41.006053] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:184) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:703996121088}, ls->get_ls_id()={id:1001})
[2023-07-23 11:04:41.006847] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] succ to get rec scn(*this={this:0x7fd46b708030, ls_id:{id:1002}, tenant_id:1002, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1689778308022019911}, last_push_gc_task_ts_:1689778309607868, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.006838] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=75][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480993499})
[2023-07-23 11:04:41.006858] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc5aa0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1689778309609228}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc5aa0, snapshot_version:{val:1689778309609228}, ls_id:{id:1002}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.006871] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689738033632736044}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689738033632736044}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1002})
[2023-07-23 11:04:41.006880] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd45c21a250}, min_rec_scn={val:1690080741745971437}, ls_id_={id:1002})
[2023-07-23 11:04:41.006871] WDIAG [SQL.OPT] strong_select_replicas (ob_log_plan.cpp:2154) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=33][errcode=-4722] fail to all select leader(ret=-4722, *phy_tbl_loc_info={table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480993499}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0})
[2023-07-23 11:04:41.006894] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2117) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=23][errcode=-4722] fail to strong select replicas(ret=-4722, local_server="192.168.1.1:1001", phy_tbl_loc_info_list.count()=1)
[2023-07-23 11:04:41.006906] WDIAG [SQL.OPT] select_replicas (ob_log_plan.cpp:2038) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=11][errcode=-4722] select replicas failed(ret=-4722)
[2023-07-23 11:04:41.006919] WDIAG [SQL.PC] get_phy_locations (ob_plan_cache_util.cpp:323) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=12][errcode=-4722] failed to select replicas(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}], exec_ctx.get_addr()="192.168.1.1:1001", phy_location_info_ptrs=[{table_location_key:18446744073709551613, ref_table_id:117, candi_tablet_locs:[{opt_tablet_loc:{partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480993499}, selected_replica_idx:-1, priority_replica_idxs:[]}], duplicate_type:0}])
[2023-07-23 11:04:41.006953] WDIAG [SQL.PC] get_phy_locations (ob_plan_set.cpp:1810) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=34][errcode=-4722] failed to get phy locations(ret=-4722, table_locations=[{loc_meta:{table_loc_id:18446744073709551613, ref_table_id:117, related_table_ids:[], use_dist_das:0, select_leader:0, is_dup_table:0, is_weak_read:1, unuse_related_pruning:0}, part_projector:{column_cnt:0, part_projector:null, subpart_projector:null}, has_dynamic_exec_param:false, part_hint_ids:[], related_list:[]}])
[2023-07-23 11:04:41.006973] WDIAG [SQL.PC] get_plan_type (ob_plan_set.cpp:2102) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=19][errcode=-4722] failed to get physical locations(ret=-4722)
[2023-07-23 11:04:41.006982] WDIAG [SQL.PC] select_plan (ob_plan_set.cpp:1347) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8][errcode=-4722] fail to get plan normal(ret=-4722)
[2023-07-23 11:04:41.006992] WDIAG [SQL] execute_get_plan (ob_sql.cpp:3513) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=7][errcode=-4722] fail to get physical plan(ret=-4722)
[2023-07-23 11:04:41.007198] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080741745971437}, checkpoint_scn_in_ls_meta={val:1690080741745971437}, ls_id={id:1002}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:41.007207] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1153) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1002, self:"192.168.1.1:1001", has_set_deleted:false}, curr_base_lsn={lsn:406789103616}, new_base_lsn={lsn:406789103616}, lsn={lsn:406789103616})
[2023-07-23 11:04:41.007215] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:184) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:406789103616}, ls->get_ls_id()={id:1002})
[2023-07-23 11:04:41.008017] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=6] succ to get rec scn(*this={this:0x7fd447204030, ls_id:{id:1003}, tenant_id:1002, state:"L_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1689838148069950986}, last_push_gc_task_ts_:1689838151799175, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.008029] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=11] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd4c5bc67b0, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1689838151801616}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd4c5bc67b0, snapshot_version:{val:1689838151801616}, ls_id:{id:1003}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.008042] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:1689738033637759566}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:1689738033637759566}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1003})
[2023-07-23 11:04:41.008051] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7fd4653e4250}, min_rec_scn={val:1690080741730241082}, ls_id_={id:1003})
[2023-07-23 11:04:41.008369] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=7] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690080741730241082}, checkpoint_scn_in_ls_meta={val:1690080741730241082}, ls_id={id:1003}, service_type="TRANS_SERVICE")
[2023-07-23 11:04:41.008378] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1153) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1003, self:"192.168.1.1:1001", has_set_deleted:false}, curr_base_lsn={lsn:375249862656}, new_base_lsn={lsn:375249862656}, lsn={lsn:375249862656})
[2023-07-23 11:04:41.008387] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:184) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:375249862656}, ls->get_ls_id()={id:1003})
[2023-07-23 11:04:41.008393] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:191) [83140][T1002_TxCkpt][T1002][Y0-0000000000000000-0-0] [lt=5] succeed to update_clog_checkpoint(ret=0, ls_cnt=4)
[2023-07-23 11:04:41.009022] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=8] total rowcount, use statistics(table_meta_info_.table_row_count_=3, table_meta_info_.average_row_size_=3.800000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=1.140000000000000000e+03)
[2023-07-23 11:04:41.009082] WDIAG [SQL.OPT] all_select_leader (ob_phy_table_location_info.cpp:448) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=25][errcode=-4722] fail to get leader(ret=-4722, phy_part_loc_info.get_partition_location()={partition_id_:117, replica_locations:[{ls_replica_location:{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:0}, attr_:{pos_type_:4, merge_status_:2, zone_type_:3, zone_status_:3, start_service_time_:0, server_stop_time_:0, server_status_:4}, is_filter_:false, replica_idx_:-1}], renew_time:1690081480993499})
[2023-07-23 11:04:41.009153] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949880855, cache_obj->added_lc()=false, cache_obj->get_object_id()=20507, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:41.009181] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=14] [packet retry query](TRACE=begin_ts=1690081481006781 2023-07-23 03:04:41.006781|[process_begin] u=0 in_queue_time:5782090, receive_ts:1690081475224690, enqueue_ts:1690081481006770|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=6 |[session] u=2 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=40 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[pc_choose_plan] u=156 |[transform_with_outline_begin] u=69 |[transform_with_outline_end] u=1 |[resolve_begin] u=3 |[resolve_end] u=683 |[transform_begin] u=24 |[optimize_begin] u=1074 |[tl_calc_part_id_end] u=166 |[get_location_cache_begin] u=0 |[get_location_cache_begin] u=118 |[query_end] u=50 |[process_end] u=1 run_ts:1690081481006789|total_timeu=2398)
[2023-07-23 11:04:41.016990] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=31][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:41.017001] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=10][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196781}, scn:1690081480993247}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196781}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480906031, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.017032] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=32][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196781}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480906031, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.017180] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=147][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:41.017187] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:41.017191] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:41.017199] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:41.017203] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:41.017208] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:41.017212] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:41.017218] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702504009, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196781}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480906031, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196781}, scn:1690081480993247}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:41.017613] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=394][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:41.017618] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:41.017627] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:41.017631] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:41.017635] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:41.017640] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:41.017646] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:41.017650] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:41.017654] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:41.017659] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:41.017663] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:41.017667] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:41.017693] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:73, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:41.017707] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196781}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480906031, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:41.017740] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=32] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196781}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480906031, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:41.017765] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081480993247, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196781}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480906031, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:41.017794] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=26][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:41.017801] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:41.017805] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:41.017812] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=92257427)
[2023-07-23 11:04:41.017827] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081481016897 2023-07-23 03:04:41.016897|[process_begin] u=0 in_queue_time:7741658, receive_ts:1690081473275239, enqueue_ts:1690081481016886|[start_sql] u=1 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=44 |[check_priv] u=8 |[plan_id] u=0 plan_id:28026|[do_open_plan_begin] u=2 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=6 |[insert_start] u=0 |[get_das_id_begin] u=11 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=231 |[sync_auto_value] u=422 |[close_plan_begin] u=63 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=86 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=32 |[process_end] u=1 run_ts:1690081481016900|total_timeu=929)
[2023-07-23 11:04:41.018152] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.018636] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.019097] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.019556] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3290][TbltLSSrv3][T1002][YB420A01819C-0005FDE64EB793FC-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.020305] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3786][PxTargetMgr0][T0][Y0-0000000000000000-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.027657] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:41.027685] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=22][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:41.027701] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:41.027714] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=9][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:41.027728] WDIAG [SQL.RESV] get_table_schema (ob_schema_checker.cpp:850) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7][errcode=-4002] invalid arguments(table_id=18446744073709551615, ret=-4002)
[2023-07-23 11:04:41.030913] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:41.032538] INFO  [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:10736) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=7] total rowcount, use statistics(table_meta_info_.table_row_count_=24, table_meta_info_.average_row_size_=1.170000000000000000e+02, table_meta_info_.micro_block_count_=1, table_meta_info_.part_size_=2.808000000000000000e+03)
[2023-07-23 11:04:41.033598] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:139) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0][errcode=0] set logical del time(cache_obj->get_logical_del_time()=3541949905300, cache_obj->added_lc()=false, cache_obj->get_object_id()=20508, cache_obj->get_tenant_id()=1, lbt()="0xdaaa2a5 0x81a5f2c 0x3a276d9 0x813a68a 0x3cc8b6f 0x3e59f7a 0x3a1a901 0x3a13e04 0x3a0fe1a 0x3a0df4b 0x3a0b1c1 0x6362ea4 0xdd1c5c7 0xdd1641a 0x7fd630b4aea5 0x7fd630873b0d")
[2023-07-23 11:04:41.033624] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=14] [packet retry query](TRACE=begin_ts=1690081481026963 2023-07-23 03:04:41.026963|[process_begin] u=0 in_queue_time:530087, receive_ts:1690081480496875, enqueue_ts:1690081481026953|[start_sql] u=1 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=2 |[session] u=2 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=32 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[pc_choose_plan] u=8 |[transform_with_outline_begin] u=39 |[transform_with_outline_end] u=1 |[resolve_begin] u=2 |[resolve_end] u=814 |[transform_begin] u=24 |[optimize_begin] u=4318 |[tl_calc_part_id_end] u=313 |[get_location_cache_begin] u=0 |[storage_estimation_begin] u=68 |[storage_estimation_end] u=943 |[get_location_cache_begin] u=32 |[query_end] u=51 |[process_end] u=1 run_ts:1690081481026967|total_timeu=6659)
[2023-07-23 11:04:41.041595] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC43-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:41.041589] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=44] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1690081481041574})
[2023-07-23 11:04:41.041600] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E360-0-0] [lt=7] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1690081481041575})
[2023-07-23 11:04:41.041607] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3291][][T0][YB420A01819C-0005FDE715FDDC43-0-0] [lt=10][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.041618] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC44-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:41.042014] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=21] new mysql sessid created(conn.sessid_=3222112529, support_ssl=false)
[2023-07-23 11:04:41.042062] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=22] sm conn init succ(conn.sessid_=3222112529, sess.client_addr_="127.0.0.1:58002")
[2023-07-23 11:04:41.042084] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3391][sql_nio13][T0][Y0-0000000000000000-0-0] [lt=16] accept one succ(*s={this:0x7fd3373df290, fd:391, err:0, last_decode_time_:0, last_write_time_:1690081481042060, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:41.042835] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=12] new mysql sessid created(conn.sessid_=3222113094, support_ssl=false)
[2023-07-23 11:04:41.042865] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=14] sm conn init succ(conn.sessid_=3222113094, sess.client_addr_="127.0.0.1:58004")
[2023-07-23 11:04:41.042877] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3378][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=9] accept one succ(*s={this:0x7fd433492030, fd:381, err:0, last_decode_time_:0, last_write_time_:1690081481042864, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:41.045586] INFO  [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:104) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=9] new mysql sessid created(conn.sessid_=3222110435, support_ssl=false)
[2023-07-23 11:04:41.045618] INFO  [RPC.OBMYSQL] init (obsm_conn_callback.cpp:120) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=13] sm conn init succ(conn.sessid_=3222110435, sess.client_addr_="127.0.0.1:58006")
[2023-07-23 11:04:41.045632] INFO  [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:899) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=8] accept one succ(*s={this:0x7fd39cc7c170, fd:392, err:0, last_decode_time_:0, last_write_time_:1690081481045617, read_buffer_.get_consume_sz():0, get_pending_flag():0, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:41.048730] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:309) [83378][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=8] ====== traversal_flush timer task ======
[2023-07-23 11:04:41.048768] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:341) [83378][T1001_Flush][T1001][Y0-0000000000000000-0-0] [lt=15] succeed to traversal_flush(ret=0, ls_cnt=1)
[2023-07-23 11:04:41.051190] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.051208] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081481051187]})
[2023-07-23 11:04:41.051222] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081481051216]})
[2023-07-23 11:04:41.051230] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:41.051240] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=10][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:41.051253] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:41.051260] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:41.051374] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1002, gts_local_cache={srr:[mts=1690081465143548], gts:1690081465143322587, latest_srr:[mts=1690081481051367]})
[2023-07-23 11:04:41.051494] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=8][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:41.051507] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=13][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:41.051520] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=8][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:41.051542] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=21][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:41.051561] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=19] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:41.051582] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=18] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:41.051591] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06B-0-0] [lt=6] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081481051589, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:41.051977] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3273][MetaLocAsyncUp0][T1][YB420A01819C-0005FDE627E7B203-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.052250] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C43-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.052278] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3274][UserLocAsyncUp0][T1001][YB420A01819C-0005FDE628679C43-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.052387] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=22][errcode=-4009] open file error(filename="cgroup/tenant_1/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:41.052419] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=34][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1/cpuacct.usage", usage_value="", tenant_id=1)
[2023-07-23 11:04:41.053193] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=13][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:133, local_retry_times:133, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:41.053209] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=14][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:41.053216] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:41.053220] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=3][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:41.053221] INFO  [SERVER] extract_user_tenant (obmp_connect.cpp:1243) [3192][MysqlQueueTh0][T0][Y0-0005FDE6FD2C2EC9-0-0] [lt=39] username and tenantname(user_name=ocp_monitor, tenant_name=)
[2023-07-23 11:04:41.053235] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:41.053284] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=1][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:41.053323] INFO  load_privilege_info (obmp_connect.cpp:426) [3192][MysqlQueueTh0][T1][Y0-0005FDE6FD2C2EC9-0-0] [lt=24] no tenant name set, use default tenant name(tenant_name=sys)
[2023-07-23 11:04:41.053972] INFO  [SERVER] process (obmp_connect.cpp:369) [3192][MysqlQueueTh0][T1][Y0-0005FDE6FD2C2EC9-0-0] [lt=20] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222110435, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:41.054300] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=26] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080742373882309}, clog_base_lsn:{lsn:8388096000}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831059622276}, all_id_meta:{id_meta:[{limited_id:1690081474308899777, latest_log_ts:{val:1690081457643664628}}, {limited_id:834000001, latest_log_ts:{val:1690081371645352093}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:41.054345] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=43] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1001}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741701524066}, clog_base_lsn:{lsn:703996121088}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831251246577}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:41.054374] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=29] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1002}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741745971437}, clog_base_lsn:{lsn:406789103616}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831214220192}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:2, proposal_id:0, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1002, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:41.054401] INFO  [CLOG] do_fetch_log_ (ob_remote_fetch_log.cpp:154) [83123][T1002_LogRessvr][T1002][YB420A01819C-0005FDE627D78DB5-0-0] [lt=26] print do_fetch_log_(lsn={lsn:18446744073709551615}, max_fetch_lsn={lsn:18446744073709551615}, need_schedule=false, proposal_id=-1, last_fetch_ts=-1, task_count=0, ls={ls_meta:{tenant_id:1002, ls_id:{id:1003}, replica_type:0, ls_create_status:1, clog_checkpoint_scn:{val:1690080741730241082}, clog_base_lsn:{lsn:375249862656}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615}, restore_status:{status:0}, replayable_point:{val:1690081463683475827}, tablet_change_checkpoint_scn:{val:1689921831245549869}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}, {limited_id:1, latest_log_ts:{val:18446744073709551615}}]}}, log_handler:{role:1, proposal_id:54, palf_env_:0x7fd4d8dde030, is_in_stop_state_:false, is_inited_:true}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1003, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}})
[2023-07-23 11:04:41.054438] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1001][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=15][errcode=-4009] open file error(filename="cgroup/tenant_1001/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:41.054459] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1001][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=22][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1001/cpuacct.usage", usage_value="", tenant_id=1001)
[2023-07-23 11:04:41.057280] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC45-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:41.057278] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=36] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1690081481057267})
[2023-07-23 11:04:41.057550] WDIAG [SERVER.OMT] get_string_from_file_ (ob_cgroup_ctrl.cpp:894) [3767][T1_L0_G0][T1002][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=13][errcode=-4009] open file error(filename="cgroup/tenant_1002/cpuacct.usage", errno=2, errmsg="No such file or directory", ret=-4009)
[2023-07-23 11:04:41.057571] WDIAG [SERVER.OMT] get_cpu_time (ob_cgroup_ctrl.cpp:678) [3767][T1_L0_G0][T1002][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=21][errcode=-4009] get cpu usage failed(ret=-4009, usage_path="cgroup/tenant_1002/cpuacct.usage", usage_value="", tenant_id=1002)
[2023-07-23 11:04:41.057636] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1001", warnings:[]}, tenant_id_=1001, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:41.057661] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=1] get wrs ts(ls_id={id:1}, delta=15596233, timestamp={val:1690081465461187621}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:41.057672] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83376][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1001, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081460674676536}", last_lsn={lsn:99688686671}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:41.058458] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.058556] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:264) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C88-0-0] [lt=13][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:41.058653] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C71-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081481057146 2023-07-23 03:04:41.057146|[process_begin] u=0 in_queue_time:5103825, receive_ts:1690081475953320, enqueue_ts:1690081481057129|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57856}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C71-0-0|[before_processor_run] u=3 |[session] u=2 sid:3222122353, tenant_id:1|[calc_partition_location_begin] u=93 |[plc_serialize_begin] u=3 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=6 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=3 |[calc_partition_location_begin] u=0 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=3 |[tl_calc_part_id_end] u=63 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=5 |[pc_choose_plan] u=2 |[check_priv] u=6 |[plan_id] u=1 plan_id:20478|[do_open_plan_begin] u=4 plan_id:20478|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=13 |[do_open_plan_end] u=0 |[get_das_id_begin] u=51 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=3 |[do_local_das_task_end] u=27 |[calc_partition_location_begin] u=956 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=4 |[get_plan_type_end] u=1 |[pc_choose_plan] u=2 |[check_priv] u=2 |[plan_id] u=1 plan_id:427|[do_open_plan_begin] u=15 plan_id:427|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[remote_task_completed] u=57 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=3 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=2 |[close_plan_begin] u=8 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=1 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[close_plan_begin] u=46 |[close_das_task_begin] u=1 |[close_das_task_end] u=44 |[start_end_stmt] u=29 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[query_end] u=27 |[process_end] u=2 run_ts:1690081481057151|total_timeu=1505)
[2023-07-23 11:04:41.064839] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1959-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.064854] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83046][T1002_TenantInf][T1001][YB420A01819C-0005FDE72DDD1959-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.065969] INFO  [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1331) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=5] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1)
[2023-07-23 11:04:41.065983] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1022) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=12] start lib cache evict(tenant_id=1, mem_hold=211812352, mem_limit=858993455, cache_obj_num=732, cache_node_num=729)
[2023-07-23 11:04:41.065991] INFO  [SQL.PC] cache_evict (ob_plan_cache.cpp:1039) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=7] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=211812352, mem_limit=858993455, cache_obj_num=732, cache_node_num=729)
[2023-07-23 11:04:41.065998] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2024) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:41.067397] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=24][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:41.067407] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=10][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196782}, scn:1690081481051100}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196782}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480956512, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.067437] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=30][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081572560135, schema_version:1687174506453632, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196782}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480956512, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.067588] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=149][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200279})
[2023-07-23 11:04:41.067595] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:41.067600] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200279}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500866)
[2023-07-23 11:04:41.067608] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:41.067613] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:41.067619] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:41.067623] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:41.067629] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081572660135, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702504010, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd4380bc2e0, tx_id:{txid:832196782}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480956512, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196782}, scn:1690081481051100}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200279}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200279}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500869, schema_version:1687170003168040, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500869, schema_version:1687170003168040, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_pnr_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500914, schema_version:1687174404276088, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500914, schema_version:1687174404276088, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500915, schema_version:1687174506453624, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500915, schema_version:1687174506453624, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847546}, {id:1152921504606847587}, {id:1152921504606847588}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500866, schema_version:1687174506453632, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:16777215, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689588820912688, data_table:{table_id:500866, schema_version:1687174506453632, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd3e608d230, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:16777215, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd3e608d230, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"MEDIUMTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081572560135, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3663, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:41.068020] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=390][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:41.068025] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:41.068033] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:41.068037] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:41.068041] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:41.068045] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:41.068051] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:41.068056] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:41.068060] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:41.068067] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:41.068071] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:41.068077] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:41.068096] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=3][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:79, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:41.068111] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196782}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480956512, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:41.068140] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=29] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, desc={this:0x7fd4380bc2e0, tx_id:{txid:832196782}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081480956512, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:41.068166] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=25] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081481051100, tx_desc={this:0x7fd4380bc2e0, tx_id:{txid:832196782}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222112438, assoc_session_id:3222112438, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081480956512, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d3a7c8, id:3222112438, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd4380bc2e0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:41.068195] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=25][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:41.068202] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=7][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:41.068206] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081572660135)
[2023-07-23 11:04:41.068214] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=6][errcode=-4038] try to execute again(ret=-4038, type=2, retry_type=2, timeout_remain=91591921)
[2023-07-23 11:04:41.068228] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57F9-0-0] [lt=5] [packet retry query](TRACE=begin_ts=1690081481067286 2023-07-23 03:04:41.067286|[process_begin] u=0 in_queue_time:8407151, receive_ts:1690081472660135, enqueue_ts:1690081481067272|[start_sql] u=1 addr:{ip:"192.168.1.1", port:41192}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE730AB57F9-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112438, tenant_id:1002|[pc_choose_plan] u=56 |[check_priv] u=8 |[plan_id] u=0 plan_id:33091|[do_open_plan_begin] u=2 plan_id:33091|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=1 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=8 |[insert_start] u=0 |[get_das_id_begin] u=14 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=236 |[sync_auto_value] u=416 |[close_plan_begin] u=61 |[insert_close] u=1 |[close_das_task_begin] u=1 |[close_das_task_end] u=0 |[start_end_stmt] u=1 |[end_stmt] u=83 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=32 |[process_end] u=1 run_ts:1690081481067289|total_timeu=941)
[2023-07-23 11:04:41.068831] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879366-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.069564] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879366-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.070254] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879366-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.069362] INFO  [SQL.PC] dump_all_objs (ob_plan_cache.cpp:1799) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=10] Dumping All Cache Objs(alloc_obj_list.count()=732, alloc_obj_list=[{obj_id:1, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109325}, {obj_id:2, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111572}, {obj_id:3, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:341289}, {obj_id:4, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235127}, {obj_id:5, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184447}, {obj_id:6, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229842}, {obj_id:7, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179161}, {obj_id:8, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123908}, {obj_id:9, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128433}, {obj_id:11, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116945}, {obj_id:13, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129603}, {obj_id:15, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117454}, {obj_id:16, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109248}, {obj_id:17, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129226}, {obj_id:18, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109760}, {obj_id:19, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122149}, {obj_id:20, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114037}, {obj_id:21, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:158335}, {obj_id:22, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121145}, {obj_id:23, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128430}, {obj_id:26, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226632}, {obj_id:29, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115569}, {obj_id:37, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145358}, {obj_id:38, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165873}, {obj_id:40, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127310}, {obj_id:41, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146874}, {obj_id:42, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131649}, {obj_id:43, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131770}, {obj_id:44, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125898}, {obj_id:45, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145121}, {obj_id:46, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130841}, {obj_id:47, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131994}, {obj_id:48, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121918}, {obj_id:49, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119714}, {obj_id:50, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118468}, {obj_id:51, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132696}, {obj_id:52, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125866}, {obj_id:53, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121229}, {obj_id:54, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118644}, {obj_id:55, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121560}, {obj_id:56, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118676}, {obj_id:57, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121565}, {obj_id:58, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118621}, {obj_id:59, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124750}, {obj_id:60, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120121}, {obj_id:61, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116266}, {obj_id:62, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141345}, {obj_id:63, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116282}, {obj_id:64, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120980}, {obj_id:65, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116682}, {obj_id:66, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124583}, {obj_id:67, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116330}, {obj_id:68, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117450}, {obj_id:69, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239802}, {obj_id:70, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187352}, {obj_id:73, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223151}, {obj_id:74, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170701}, {obj_id:76, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113692}, {obj_id:78, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125220}, {obj_id:81, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116890}, {obj_id:82, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115894}, {obj_id:84, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111136}, {obj_id:112, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121899}, {obj_id:113, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159037}, {obj_id:114, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118845}, {obj_id:116, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88994}, {obj_id:117, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151029}, {obj_id:118, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117628}, {obj_id:119, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118807}, {obj_id:120, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88832}, {obj_id:121, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:124, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109685}, {obj_id:135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170252}, {obj_id:139, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:157877}, {obj_id:152, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133335}, {obj_id:155, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113973}, {obj_id:156, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195192}, {obj_id:157, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119284}, {obj_id:161, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109728}, {obj_id:162, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108544}, {obj_id:164, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115837}, {obj_id:165, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111259}, {obj_id:167, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137904}, {obj_id:171, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118032}, {obj_id:172, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117558}, {obj_id:173, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:306152}, {obj_id:174, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108552}, {obj_id:175, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112192}, {obj_id:176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109808}, {obj_id:177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111256}, {obj_id:178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112154}, {obj_id:179, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112278}, {obj_id:180, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147332}, {obj_id:181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120509}, {obj_id:184, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112479}, {obj_id:185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118190}, {obj_id:186, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225682}, {obj_id:187, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128216}, {obj_id:189, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120694}, {obj_id:190, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152472}, {obj_id:192, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124689}, {obj_id:194, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121903}, {obj_id:195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107757}, {obj_id:197, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110196}, {obj_id:198, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106208}, {obj_id:200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126038}, {obj_id:201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120794}, {obj_id:202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112445}, {obj_id:203, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138373}, {obj_id:204, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124081}, {obj_id:206, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110192}, {obj_id:213, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:373654}, {obj_id:214, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:505828}, {obj_id:215, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113398}, {obj_id:220, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131588}, {obj_id:221, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144772}, {obj_id:230, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107418}, {obj_id:232, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115072}, {obj_id:233, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:234, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:93529}, {obj_id:235, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123553}, {obj_id:236, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116626}, {obj_id:238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121199}, {obj_id:239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106928}, {obj_id:240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112090}, {obj_id:264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:90440}, {obj_id:266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135852}, {obj_id:267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134273}, {obj_id:269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145191}, {obj_id:273, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123608}, {obj_id:274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108417}, {obj_id:278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111783}, {obj_id:305, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121676}, {obj_id:306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141832}, {obj_id:311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129452}, {obj_id:312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115712}, {obj_id:313, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124673}, {obj_id:322, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88886}, {obj_id:324, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140374}, {obj_id:328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113091}, {obj_id:330, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159280}, {obj_id:331, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125300}, {obj_id:332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117330}, {obj_id:337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123557}, {obj_id:345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111769}, {obj_id:346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123643}, {obj_id:356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125880}, {obj_id:361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112752}, {obj_id:362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110232}, {obj_id:363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108568}, {obj_id:364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113753}, {obj_id:365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119640}, {obj_id:402, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:857901}, {obj_id:403, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:452223}, {obj_id:411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110518}, {obj_id:421, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121605}, {obj_id:422, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128289}, {obj_id:426, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:128217}, {obj_id:427, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107111}, {obj_id:429, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:108426}, {obj_id:437, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117618}, {obj_id:438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138464}, {obj_id:439, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129287}, {obj_id:440, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:105488}, {obj_id:443, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115530}, {obj_id:447, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197761}, {obj_id:450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199814}, {obj_id:452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207533}, {obj_id:474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133309}, {obj_id:522, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:375064}, {obj_id:524, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:480101}, {obj_id:534, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117040}, {obj_id:580, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122952}, {obj_id:605, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:345007}, {obj_id:660, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223440}, {obj_id:661, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115072}, {obj_id:665, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118024}, {obj_id:666, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116400}, {obj_id:667, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114060}, {obj_id:802, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120968}, {obj_id:855, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121960}, {obj_id:862, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114399}, {obj_id:863, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118344}, {obj_id:869, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117119}, {obj_id:876, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150075}, {obj_id:877, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175815}, {obj_id:925, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195664}, {obj_id:933, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129181}, {obj_id:944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132503}, {obj_id:945, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137742}, {obj_id:946, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126636}, {obj_id:948, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120424}, {obj_id:955, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127224}, {obj_id:956, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116704}, {obj_id:957, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:114720}, {obj_id:1014, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173857}, {obj_id:1015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132988}, {obj_id:1016, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129138}, {obj_id:1083, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124264}, {obj_id:1145, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130168}, {obj_id:1146, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122296}, {obj_id:1169, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280049}, {obj_id:1266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88881}, {obj_id:1269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:11552}, {obj_id:1271, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:916091}, {obj_id:1274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:1275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107490}, {obj_id:1276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110398}, {obj_id:1278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124488}, {obj_id:1279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121664}, {obj_id:1283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106461}, {obj_id:1284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142870}, {obj_id:1285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175250}, {obj_id:1286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161389}, {obj_id:1287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137813}, {obj_id:1288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208605}, {obj_id:1289, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156033}, {obj_id:1290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159046}, {obj_id:1291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191322}, {obj_id:1297, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:144114}, {obj_id:1298, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131804}, {obj_id:1299, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167182}, {obj_id:1300, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140586}, {obj_id:1301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237139}, {obj_id:1302, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162351}, {obj_id:1303, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220896}, {obj_id:1306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143476}, {obj_id:1307, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164310}, {obj_id:1308, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121121}, {obj_id:1309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220799}, {obj_id:1310, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142485}, {obj_id:1311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204086}, {obj_id:1315, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153942}, {obj_id:1316, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132418}, {obj_id:1317, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213169}, {obj_id:1318, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176390}, {obj_id:1319, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:428742}, {obj_id:1320, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208446}, {obj_id:1321, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:420997}, {obj_id:1325, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:147492}, {obj_id:1326, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182923}, {obj_id:1327, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:811052}, {obj_id:1328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:300175}, {obj_id:1329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:967217}, {obj_id:1330, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286896}, {obj_id:1334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143185}, {obj_id:1335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131501}, {obj_id:1336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162857}, {obj_id:1337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:411417}, {obj_id:1338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216472}, {obj_id:1339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:488389}, {obj_id:1340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199461}, {obj_id:1345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145055}, {obj_id:1346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169934}, {obj_id:1347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126655}, {obj_id:1348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248008}, {obj_id:1349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:149096}, {obj_id:1350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238697}, {obj_id:1353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132741}, {obj_id:1354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171395}, {obj_id:1355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251238}, {obj_id:1356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165610}, {obj_id:1359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145342}, {obj_id:1360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:172824}, {obj_id:1361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257085}, {obj_id:1362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:241945}, {obj_id:1367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146300}, {obj_id:1368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177171}, {obj_id:1369, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274985}, {obj_id:1370, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260698}, {obj_id:1372, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:466085}, {obj_id:1373, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:554863}, {obj_id:1380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132111}, {obj_id:1381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:148619}, {obj_id:1382, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175303}, {obj_id:1385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143827}, {obj_id:1386, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165757}, {obj_id:1387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132159}, {obj_id:1388, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226691}, {obj_id:1389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:155636}, {obj_id:1390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210276}, {obj_id:1400, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:153845}, {obj_id:1522, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132816}, {obj_id:1538, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:296281}, {obj_id:1641, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:146892}, {obj_id:1642, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178588}, {obj_id:1643, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283433}, {obj_id:1644, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275590}, {obj_id:1645, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142583}, {obj_id:1648, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:143372}, {obj_id:1669, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:468828}, {obj_id:1686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:152827}, {obj_id:1781, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:380248}, {obj_id:1782, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:517009}, {obj_id:1841, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117704}, {obj_id:2349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:122600}, {obj_id:2362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:111073}, {obj_id:2363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112556}, {obj_id:2365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:140294}, {obj_id:2366, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133478}, {obj_id:2394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260616}, {obj_id:2431, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:117352}, {obj_id:2451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121272}, {obj_id:2452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201000}, {obj_id:2491, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151697}, {obj_id:2500, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135218}, {obj_id:2502, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145948}, {obj_id:2503, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175710}, {obj_id:2504, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269768}, {obj_id:2505, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:255176}, {obj_id:2515, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180033}, {obj_id:2516, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182644}, {obj_id:2517, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:291295}, {obj_id:2518, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225595}, {obj_id:2519, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141982}, {obj_id:2535, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163630}, {obj_id:2536, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193037}, {obj_id:2538, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:558139}, {obj_id:2561, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156119}, {obj_id:2687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:123272}, {obj_id:2712, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:335251}, {obj_id:2864, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:357639}, {obj_id:2868, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:453965}, {obj_id:2924, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:139696}, {obj_id:3119, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124568}, {obj_id:3201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:134402}, {obj_id:3202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137472}, {obj_id:3203, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132350}, {obj_id:3275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138117}, {obj_id:3276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127600}, {obj_id:3277, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107922}, {obj_id:3279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110460}, {obj_id:3280, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116675}, {obj_id:3283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:3284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118296}, {obj_id:3285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:137904}, {obj_id:3286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112502}, {obj_id:3288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:113940}, {obj_id:3290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106432}, {obj_id:3291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:110392}, {obj_id:3293, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:118680}, {obj_id:3294, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107219}, {obj_id:3295, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127909}, {obj_id:3309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:390507}, {obj_id:3473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:130856}, {obj_id:3480, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151736}, {obj_id:3498, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:319008}, {obj_id:3748, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:138080}, {obj_id:3749, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:115376}, {obj_id:3938, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125896}, {obj_id:3993, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:135764}, {obj_id:4060, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249784}, {obj_id:4061, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:109801}, {obj_id:4062, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:97597}, {obj_id:4063, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:4064, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4065, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:4066, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4067, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88838}, {obj_id:4068, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4069, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4072, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4073, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88839}, {obj_id:4074, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4075, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4079, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:133294}, {obj_id:4080, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88833}, {obj_id:4081, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:107168}, {obj_id:4084, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88964}, {obj_id:4195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:333559}, {obj_id:4200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:416289}, {obj_id:4267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:131792}, {obj_id:4268, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:116016}, {obj_id:4301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:286580}, {obj_id:4518, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126536}, {obj_id:4546, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:92379}, {obj_id:4547, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4548, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4549, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88839}, {obj_id:4550, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4551, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106960}, {obj_id:4555, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:112431}, {obj_id:4556, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:106944}, {obj_id:4568, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:299521}, {obj_id:4677, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173951}, {obj_id:4678, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202163}, {obj_id:4683, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185458}, {obj_id:4684, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219118}, {obj_id:4687, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:792994}, {obj_id:4691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:805593}, {obj_id:4692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:960692}, {obj_id:4726, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150855}, {obj_id:4866, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:88952}, {obj_id:5048, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:259342}, {obj_id:5051, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:302866}, {obj_id:5079, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:125592}, {obj_id:5128, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136112}, {obj_id:5287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:340096}, {obj_id:5291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:426188}, {obj_id:5364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:127560}, {obj_id:5380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120632}, {obj_id:5381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:124904}, {obj_id:5559, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:359785}, {obj_id:5561, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:456815}, {obj_id:5615, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257334}, {obj_id:5651, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:136416}, {obj_id:5652, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174265}, {obj_id:5653, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:129528}, {obj_id:5884, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234650}, {obj_id:5909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177506}, {obj_id:6176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:142016}, {obj_id:6177, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:151189}, {obj_id:6178, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:120328}, {obj_id:6208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126232}, {obj_id:6385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274624}, {obj_id:6387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:329196}, {obj_id:6470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171259}, {obj_id:6578, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190649}, {obj_id:6579, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215440}, {obj_id:6584, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193411}, {obj_id:6590, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205876}, {obj_id:6631, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:156453}, {obj_id:6774, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174901}, {obj_id:7027, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:171911}, {obj_id:7060, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263837}, {obj_id:7191, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:344496}, {obj_id:7195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:433143}, {obj_id:7208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:121656}, {obj_id:7263, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:132480}, {obj_id:7264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:150203}, {obj_id:7450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:278957}, {obj_id:7452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:332196}, {obj_id:7459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250592}, {obj_id:7462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290128}, {obj_id:7486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:126920}, {obj_id:7547, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:170957}, {obj_id:7806, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:748217}, {obj_id:7807, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:889404}, {obj_id:7813, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:904154}, {obj_id:7814, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1074659}, {obj_id:7882, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:162719}, {obj_id:7921, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283317}, {obj_id:8073, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272428}, {obj_id:8075, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:322991}, {obj_id:8143, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:141376}, {obj_id:8144, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:145305}, {obj_id:8367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:346670}, {obj_id:8368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:436340}, {obj_id:8449, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167665}, {obj_id:8450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:119976}, {obj_id:8483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276813}, {obj_id:8675, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270240}, {obj_id:8676, tenant_id:1, log_del_time:9223372036854775807, real_d
[2023-07-23 11:04:41.070383] INFO  [SQL.PC] runTimerTask (ob_plan_cache.cpp:2032) [3639][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=1020] schedule next cache evict task(evict_interval=1000000)
[2023-07-23 11:04:41.070530] INFO  [SERVER] extract_user_tenant (obmp_connect.cpp:1243) [3194][MysqlQueueTh2][T0][Y0-0005FDE6FC4C4545-0-0] [lt=16] username and tenantname(user_name=ocp_monitor, tenant_name=)
[2023-07-23 11:04:41.070598] INFO  load_privilege_info (obmp_connect.cpp:426) [3194][MysqlQueueTh2][T1][Y0-0005FDE6FC4C4545-0-0] [lt=19] no tenant name set, use default tenant name(tenant_name=sys)
[2023-07-23 11:04:41.070947] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3289][TbltLSSrv2][T1002][YB420A01819C-0005FDE660879366-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.071070] INFO  [SERVER] process (obmp_connect.cpp:369) [3194][MysqlQueueTh2][T1][Y0-0005FDE6FC4C4545-0-0] [lt=11] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222113094, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:41.072445] INFO  [SERVER] extract_user_tenant (obmp_connect.cpp:1243) [3196][MysqlQueueTh4][T0][Y0-0005FDE6FBCC4EF8-0-0] [lt=33] username and tenantname(user_name=ocp_monitor, tenant_name=)
[2023-07-23 11:04:41.072530] INFO  load_privilege_info (obmp_connect.cpp:426) [3196][MysqlQueueTh4][T1][Y0-0005FDE6FBCC4EF8-0-0] [lt=22] no tenant name set, use default tenant name(tenant_name=sys)
[2023-07-23 11:04:41.073166] INFO  [SERVER] process (obmp_connect.cpp:369) [3196][MysqlQueueTh4][T1][Y0-0005FDE6FBCC4EF8-0-0] [lt=16] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222112529, proxy_sessid=0, sess_create_time=0, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0)
[2023-07-23 11:04:41.076300] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:260) [83385][T1001_ArcSender][T1001][YB420A01819C-0005FDE628D78DB5-0-0] [lt=16][errcode=-4018] try free send task failed(ret=-4018)
[2023-07-23 11:04:41.079389] WDIAG [RPC.FRAME] create_session (ob_req_transport.cpp:263) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B86-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.079422] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [83032][T1002_Occam][T1][YB420A01819C-0005FDE72F5A9B86-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.080239] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [83145][T1002_LSArchive][T1002][YB420A01819C-0005FDE628078DB5-0-0] [lt=13] gc stale ls task succ
[2023-07-23 11:04:41.080281] INFO  [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1140) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=11] Wash compute wash size(is_wash_valid=true, sys_total_wash_size=-73134578892, global_cache_size=18789335040, tenant_max_wash_size=16646144, tenant_min_wash_size=16646144, tenant_ids_=[512, 500, 506, 508, 509, 510, 1, 1001, 1002])
[2023-07-23 11:04:41.081282] INFO  [COMMON] replace_fragment_node (ob_kvcache_map.cpp:697) [3162][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=15] Cache replace map node details(ret=0, replace_node_count=0, replace_time=17581, replace_start_pos=22900000, replace_num=100000)
[2023-07-23 11:04:41.082232] INFO  [COMMON] wash (ob_kvcache_store.cpp:343) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=36] Wash time detail, (compute_wash_size_time=134, refresh_score_time=1125, wash_time=789)
[2023-07-23 11:04:41.088013] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.088123] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C72-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081481087456 2023-07-23 03:04:41.087456|[process_begin] u=0 in_queue_time:5063629, receive_ts:1690081476023826, enqueue_ts:1690081481087445|[start_sql] u=0 addr:{ip:"127.0.0.1", port:57450}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE72D1C9C72-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112959, tenant_id:1|[calc_partition_location_begin] u=34 |[plc_serialize_begin] u=1 |[plc_serialize_end] u=4 |[tl_calc_part_id_end] u=4 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=0 plan_id:20480|[do_open_plan_begin] u=2 plan_id:20480|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_das_id_begin] u=3 |[get_das_id_end] u=0 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=21 |[calc_partition_location_begin] u=445 |[tl_calc_part_id_end] u=1 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=1 |[plan_id] u=0 plan_id:17904|[do_open_plan_begin] u=7 plan_id:17904|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[remote_task_completed] u=30 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:18446744073709551615}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=2 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=1 |[close_plan_begin] u=4 |[start_end_stmt] u=1 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=1 |[close_plan_begin] u=52 |[close_das_task_begin] u=1 |[close_das_task_end] u=24 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=1 |[auto_end_plan_end] u=0 |[query_end] u=9 |[process_end] u=0 run_ts:1690081481087459|total_timeu=666)
[2023-07-23 11:04:41.091344] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:98, local_retry_times:98, err_:-4283, err_:"OB_GTS_NOT_READY", retry_type:1, client_ret:-4283}, need_retry=true)
[2023-07-23 11:04:41.091360] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:218) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:41.091367] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:198) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] result set close failed(ret=-4283)
[2023-07-23 11:04:41.091370] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:788) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] failed to close result(close_ret=-4283, ret=-4283)
[2023-07-23 11:04:41.091383] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:815) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4283] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, record_ret=-4283, ret=-4283)
[2023-07-23 11:04:41.091432] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4122] create session fail(ret=-4122)
[2023-07-23 11:04:41.091438] WDIAG [RPC.OBRPC] rpc_post (ob_rpc_proxy.ipp:590) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=6][errcode=-4122] create request fail(ret=-4122)
[2023-07-23 11:04:41.091443] WDIAG [STORAGE.TRANS] post (ob_gts_rpc.cpp:248) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=3][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", server="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081481091429], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:41.091452] WDIAG [STORAGE.TRANS] query_gts_ (ob_gts_source.cpp:562) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4122] post gts request failed(ret=-4122, ret="OB_RPC_POST_ERROR", leader="10.1.129.154:1001", msg={tenant_id:1, srr:[mts=1690081481091429], range_size:1, sender:"192.168.1.1:1001"})
[2023-07-23 11:04:41.091461] WDIAG [STORAGE.TRANS] get_gts (ob_gts_source.cpp:251) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4023] query gts fail(tmp_ret=-4122, leader="10.1.129.154:1001")
[2023-07-23 11:04:41.093092] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=1][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1002", warnings:[]}, tenant_id_=1002, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:41.093122] INFO  [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:730) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=0] get_max_applied_scn(scn={val:1}, this={ls_id_:{id:1}, role_:2, proposal_id_:-1, palf_committed_end_lsn_:{lsn:0}, last_check_scn_:{val:18446744073709551615}, max_applied_cb_scn_:{val:1}})
[2023-07-23 11:04:41.093144] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1}, delta=15577923, timestamp={val:1690081465515197326}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:41.093158] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465515197326}", last_lsn={lsn:8392989532}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:41.093177] WDIAG [STORAGE.TRANS] process_cluster_heartbeat_rpc_cb (ob_tenant_weak_read_service.cpp:438) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0][errcode=-4016] tenant weak read service cluster heartbeat RPC fail(rcode={code:-4122, msg:"post cluster heartbeat rpc failed, tenant_id=1", warnings:[]}, tenant_id_=1, dst="192.168.1.1:1001", cluster_service_tablet_id={id:226})
[2023-07-23 11:04:41.093182] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=17] get wrs ts(ls_id={id:1001}, delta=15686370, timestamp={val:1690081465406750436}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:41.093192] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=9] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1001}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406750436}", last_lsn={lsn:704197620430}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:41.093202] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=0] get wrs ts(ls_id={id:1}, delta=15686608, timestamp={val:1690081465406512168}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:41.093211] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1002}, delta=16015657, timestamp={val:1690081465077463222}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:41.093211] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [3728][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1, LS_ID={id:1}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081465406512168}", last_lsn={lsn:24599326849}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:41.093222] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1002}, Not_Master_Cnt=1, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081464184835885}", last_lsn={lsn:406891759441}, last_gts={val:0}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:41.093240] INFO  [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:175) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=12] get wrs ts(ls_id={id:1003}, delta=15949798, timestamp={val:1690081465143322587}, min_tx_service_ts={val:4611686018427387903})
[2023-07-23 11:04:41.093251] INFO  [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:211) [83139][T1002_TenantWea][T1002][Y0-0000000000000000-0-0] [lt=10] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1002, LS_ID={id:1003}, Not_Master_Cnt=0, Near_To_GTS_Cnt=0, Other_Error_Cnt=0, Submit_Succ_Cnt=0, last_scn="{val:1690081462034732456}", last_lsn={lsn:375365134172}, last_gts={val:1690081465143322587}, min_start_scn="{val:0}", min_start_status=1)
[2023-07-23 11:04:41.104978] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC46-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:41.104970] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C89-0-0] [lt=13] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1690081481104957})
[2023-07-23 11:04:41.105456] INFO  [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:537) [83382][T1001_LSArchive][T1001][YB420A01819C-0005FDE628A78DB5-0-0] [lt=11] gc stale ls task succ
[2023-07-23 11:04:41.107723] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.107764] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C70-0-0] [lt=0] [packet retry query](TRACE=begin_ts=1690081481107647 2023-07-23 03:04:41.107647|[process_begin] u=0 in_queue_time:5882957, receive_ts:1690081475224690, enqueue_ts:1690081481107638|[start_sql] u=1 addr:{ip:"127.0.0.1", port:57898}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C70-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222112826, tenant_id:1|[calc_partition_location_begin] u=29 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=2 |[get_plan_type_end] u=1 |[pc_choose_plan] u=1 |[check_priv] u=3 |[plan_id] u=1 plan_id:17927|[do_open_plan_begin] u=1 plan_id:17927|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=1 |[exec_plan_begin] u=0 |[parse_job_begin] u=1 |[parse_job_end] u=4 |[remote_task_completed] u=40 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29269}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=11 |[do_open_plan_end] u=0 |[get_location_cache_begin] u=2 |[close_plan_begin] u=1 |[start_end_stmt] u=0 |[end_stmt] u=1 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=10 |[process_end] u=1 run_ts:1690081481107650|total_timeu=116)
[2023-07-23 11:04:41.109088] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=14][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:41.109097] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081495099847, now=[mts=1690081481053277], now0=[mts=1690081481053277], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:41.109105] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=8][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:137, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.109128] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=23][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600})
[2023-07-23 11:04:41.109141] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:137, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.109162] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=20] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd326410a98, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081481051099, tx_desc={this:0x7fd4e4c7b600, tx_id:{txid:149026361}, state:2, addr:"192.168.1.1:1001", tenant_id:1001, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:1690081465169272, op_sn:137, alloc_ts:1690081465169272, active_ts:1690081465169272, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1690081495169265, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, ref:2}, plan_type=2, stmt_type=1, has_for_update=true, query_start_time=1690081465169851, use_das=false, nested_level=0, session={this:0x7fd51afe60d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1001, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4e4c7b600}, plan=0x7fd3ede74050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:41.109198] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=35][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:370, schema_version:1686557616951712, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:41.109210] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=12][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:41.109214] WDIAG [SQL] open (ob_result_set.cpp:157) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:41.109218] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:41.109223] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"})
[2023-07-23 11:04:41.109231] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=6][errcode=-4283] execute failed(ret=-4283, tenant_id=1001, executor={ObIExecutor:, sql:"select * from __all_ls_recovery_stat where ls_id = 1003 and tenant_id = 1002 for update"}, retry_cnt=134, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686557976340344)
[2023-07-23 11:04:41.109240] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [83045][T1002_TeRec][T1001][YB420A01819C-0005FDE7302C168B-0-0] [lt=9] will sleep(sleep_us=100000, remain_us=14060608, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=134, timeout_timestamp=1690081495169847)
[2023-07-23 11:04:41.111432] INFO  [COMMON] clean_garbage_node (ob_kvcache_map.cpp:647) [3161][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=15] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=29183, clean_start_pos=9800000, clean_num=200000)
[2023-07-23 11:04:41.114821] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=1][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.114833] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:41.114841] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.154:1001", timeout=2000000, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:41.114854] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2023-07-23 11:04:41.114860] WDIAG [SHARE.PT] do_detect_master_rs_ls_ (ob_rpc_ls_table.cpp:293) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=6][errcode=0] fail to send rpc(tmp_ret=-4122, tmp_ret="OB_RPC_POST_ERROR", cluster_id=3, addr="10.1.129.155:1001", timeout=2000000, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:41.115025] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=7][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.154:1001", cluster_id:3})
[2023-07-23 11:04:41.115036] WDIAG [RPC] wait (ob_async_rpc_proxy.h:419) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=10][errcode=0] execute rpc failed(rc=-4012, server="0.0.0.0:0", timeout=0, packet code=330, arg={addr:"10.1.129.155:1001", cluster_id:3})
[2023-07-23 11:04:41.115052] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=8][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:41.115075] WDIAG [SHARE.PT] find_leader (ob_ls_info.cpp:552) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=22][errcode=-4018] fail to get leader replica(ret=-4018, ret="OB_ENTRY_NOT_EXIST", find_index=0, role=2, proposal_id_=2, *this={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:41.115095] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:140) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=20] leader doesn't exist, try use all_server_list(tmp_ret=-4018, tmp_ret="OB_ENTRY_NOT_EXIST", ls_info={tenant_id:1, ls_id:{id:1}, replicas:[{modify_time_us:0, create_time_us:0, tenant_id:1, ls_id:{id:1}, server:"192.168.1.1:1001", sql_port:2881, role:2, member_list:[{server:"10.1.129.154:1001", timestamp:1}, {server:"10.1.129.155:1001", timestamp:1}, {server:"192.168.1.1:1001", timestamp:1}], replica_type:0, proposal_id:2, replica_status:"NORMAL", restore_status:{status:0}, property:{memstore_percent_:100}, unit_id:3, zone:"zone3", paxos_replica_number:3, data_size:0, required_size:0, in_member_list:false, member_time_us:0}]})
[2023-07-23 11:04:41.115115] INFO  [SHARE.PT] get_ls_info_ (ob_rpc_ls_table.cpp:151) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=17] server_list is empty, do nothing(ret=0, ret="OB_SUCCESS", server_list=[])
[2023-07-23 11:04:41.115130] INFO  [SHARE.LOCATION] merge_leader_from (ob_location_struct.cpp:478) [3284][AutoLSLoc][T0][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=6] [LS_LOCATION] leader will be changed(new_leader={server:"10.1.129.154:1001", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}, old_leader=NULL, exist_replica=NULL, old_location={cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:3}, renew_time:1690081481115125, replica_locations:[{server:"192.168.1.1:1001", role:2, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]})
[2023-07-23 11:04:41.115614] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3284][AutoLSLoc][T1][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.116096] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3284][AutoLSLoc][T1001][YB420A01819C-0005FDE7200AAFA9-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.116466] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC47-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:41.116458] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C8A-0-0] [lt=23] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1690081481116448})
[2023-07-23 11:04:41.117824] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD9-0-0] [lt=0][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.117847] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3621][T1_Occam][T1][YB420A01819C-0005FDE72DEDDFD9-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.127958] WDIAG [STORAGE.TRANS] get_write_store_ctx (ob_trans_service_v4.cpp:1063) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=26][errcode=-4038] acquire ls snapshot for mvcc write fail(ret=-4038, ls_id={id:1003})
[2023-07-23 11:04:41.127973] WDIAG [STORAGE] get_write_store_ctx_guard_ (ob_access_service.cpp:375) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=14][errcode=-4038] can not get write store ctx(ret=-4038, ls_id={id:1003}, snapshot={this:0x7fd039ac9418, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196783}, scn:1690081481114163}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196783}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481017420, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.128003] WDIAG [STORAGE] check_write_allowed_ (ob_access_service.cpp:536) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=30][errcode=-4038] get write store ctx failed(ret=-4038, ls_id={id:1003}, dml_param={timeout:1690081573175239, schema_version:1687221732366656, sql_mode:14680064, is_total_quantity_log:true, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, tenant_schema_version:1689916332790784, is_ignore:false, prelock:false, encrypt_meta:[], is_batch_stmt:false, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0}}, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196783}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481017420, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.128153] WDIAG [STORAGE] insert_rows (ob_access_service.cpp:711) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=150][errcode=-4038] fail to check query allowed(ret=-4038, ls_id={id:1003}, tablet_id={id:200282})
[2023-07-23 11:04:41.128160] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:64) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:41.128165] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:313) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] write rows failed(ret=-4038, ls_id_={id:1003}, tablet_id_={id:200282}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=500878)
[2023-07-23 11:04:41.128173] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:114) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=8][errcode=-4038] insert rows to access service failed(ret=-4038)
[2023-07-23 11:04:41.128176] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] fail to do insert(ret=-4038)
[2023-07-23 11:04:41.128183] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:188) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] open das task op failed(ret=-4038)
[2023-07-23 11:04:41.128187] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:356) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] start local das task failed(ret=-4038)
[2023-07-23 11:04:41.128191] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:185) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] do local das task failed(ret=-4038, task_arg={timeout_ts:1690081573275239, ctrl_svr:"192.168.1.1:1001", runner_svr:"192.168.1.1:1001", task_ops:[{parent:{tenant_id:1002, task_id:3702504011, op_type:2, errcode:-4038, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:1, need_switch_param:0, trans_desc:{this:0x7fd43819eff0, tx_id:{txid:832196783}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481017420, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, snapshot:{this:0x7fd039ace628, valid:true, source:5, core:{version:{val:18446744073709551615}, tx_id:{txid:832196783}, scn:1690081481114163}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, tablet_id:{id:200282}, ls_id:{id:1003}, tablet_loc:{tablet_id:{id:200282}, ls_id:{id:1003}, server:"192.168.1.1:1001", need_refresh:0}, related_ctdefs:[{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500881, schema_version:1687170006996208, rowkey_cnt:2, spk_cnt:1, column_ids:[22, 32783, 16], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[6, 0, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500881, schema_version:1687170006996208, table_type:5, index_type:2, index_status:2, shadow_rowkey_column_num:1, fulltext_col_id:18446744073709551615, index_name:"UK_hsd_cki_xml_data_OBPK_INCRMT", pk_name:"", columns:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 4], memtable_cols_index:[0, 1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:32783, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:true}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32783 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:true, has_:true, has_shadow_:true}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500918, schema_version:1687221632220832, rowkey_cnt:2, spk_cnt:0, column_ids:[19, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:100, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[3, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500918, schema_version:1687221632220832, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_pnr", pk_name:"", columns:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500919, schema_version:1687221732366632, rowkey_cnt:2, spk_cnt:0, column_ids:[20, 16], column_types:[{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:200, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[4, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500919, schema_version:1687221732366632, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"idx_stamp", pk_name:"", columns:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:[0, 1], memtable_cols_index:[0, 1], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:3, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:2, seq_read_column_count:5, max_col_index:4, is_oracle_mode:false, cols_index:[0, 1, 2, 3, 4], memtable_cols_index:[0, 1, -1, -1, 2], cols_desc:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:4, col_cnt:5, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}], related_rtdefs:[{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}], task_status:1, related_tablet_ids:[{id:1152921504606847555}, {id:1152921504606847591}, {id:1152921504606847592}], das_task_node:}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2}, table_id:18446744073709551614, index_tid:500878, schema_version:1687221732366656, rowkey_cnt:1, spk_cnt:0, column_ids:[16, 17, 18, 19, 20, 21, 22], column_types:[{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, {type:"INT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}], column_accuracys:[{length:-1, precision:20, scale:0}, {length:-1, precision:19, scale:0}, {length:-1, precision:11, scale:0}, {length:100, precision:-1, scale:-1}, {length:200, precision:-1, scale:-1}, {length:65535, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}], old_row_projector:[], new_row_projector:[0, 1, 2, 3, 4, 5, 6], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1689315697778552, data_table:{table_id:500878, schema_version:1687221732366656, table_type:3, index_type:13, index_status:8, shadow_rowkey_column_num:0, fulltext_col_id:18446744073709551615, index_name:"", pk_name:"", columns:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], read_info:{schema_column_count:7, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:-1, seq_read_column_count:1, max_col_index:8, is_oracle_mode:false, cols_index:[0, 3, 4, 5, 6, 7, 8], memtable_cols_index:[0, 1, 2, 3, 4, 5, 6], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:9, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:[{column_id:16, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:17, meta_type:{type:"DATETIME", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:19, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:18, meta_type:{type:"INT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:11, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:19, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:20, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:200, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:21, meta_type:{type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65535, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}, {column_id:22, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false}], index_read_info:{schema_column_count:2, schema_rowkey_cnt:1, rowkey_cnt:3, trans_col_index:1, seq_read_column_count:4, max_col_index:3, is_oracle_mode:false, cols_index:[0, 1, 2, 3], memtable_cols_index:[0, -1, -1, 1], cols_desc:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=0], datum_utils:{is_oracle_mode:false, rowkey_cnt:3, col_cnt:4, allocator:0x7fd420e76bb0, is_inited:true}, cols_param:null, index_read_info:NULL}}}, col_descs:[column_id=16 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"DATETIME", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"INT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"TEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[]}, table_rowkey_cids:[], table_rowkey_types:[]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2}, timeout_ts:1690081573175239, sql_mode:14680064, prelock:false, tenant_schema_version:1689916332790784, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0}, insert_buffer:{mem_attr:tenant_id=1002, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:3044, buffer_size:1, datum_store:NULL}}], remote_info:NULL})
[2023-07-23 11:04:41.128585] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:110) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=393][errcode=-4038] failed to execute dist das task(ret=-4038)
[2023-07-23 11:04:41.128590] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:265) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to execute aggregated das task(ret=-4038, ret="OB_NOT_MASTER", aggregated_task={server:"192.168.1.1:1001", high_priority_tasks_.get_size():0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0}, async=true)
[2023-07-23 11:04:41.128598] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:297) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] failed das aggreagted task cannot retry.(ret=-4038)
[2023-07-23 11:04:41.128602] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1085) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] execute all dml das task failed(ret=-4038)
[2023-07-23 11:04:41.128605] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1175) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] failed to submit the remaining dml tasks(ret=-4038)
[2023-07-23 11:04:41.128609] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1001) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] inner get next row failed(ret=-4038, type=24, op="PHY_INSERT", op_id=0)
[2023-07-23 11:04:41.128616] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:136) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] get next row from operator failed(ret=-4038)
[2023-07-23 11:04:41.128620] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:374) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] get next row from exec result failed(ret=-4038)
[2023-07-23 11:04:41.128626] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1408) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] do dml query failed(ret=-4038)
[2023-07-23 11:04:41.128629] WDIAG [SQL] open_result (ob_result_set.cpp:213) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] fail to drive dml query(ret=-4038)
[2023-07-23 11:04:41.128636] WDIAG [SQL] open (ob_result_set.cpp:159) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=7][errcode=-4038] open result set failed(ret=-4038)
[2023-07-23 11:04:41.128640] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:62) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=3][errcode=-4038] failed to do result set open(ret=-4038)
[2023-07-23 11:04:41.128661] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:872) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=5][errcode=-4038] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:74, local_retry_times:0, err_:-4038, err_:"OB_NOT_MASTER", retry_type:2, client_ret:-4038}, need_retry=true)
[2023-07-23 11:04:41.128676] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:658) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=9] try unregister deadlock detecotr cause conflict array is empty(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196783}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481017420, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:41.128704] INFO  [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:679) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=28] maintain deadlock info(step=8, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4038, exec_ctx.get_errcode()="OB_NOT_MASTER", session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, desc={this:0x7fd43819eff0, tx_id:{txid:832196783}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481017420, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[])
[2023-07-23 11:04:41.128730] INFO  [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:923) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=25] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1690081481114163, tx_desc={this:0x7fd43819eff0, tx_id:{txid:832196783}, state:1, addr:"192.168.1.1:1001", tenant_id:1002, session_id:3222121549, assoc_session_id:3222121549, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:3, alloc_ts:1690081481017420, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7fd4f8d49428, id:3222121549, tenant:"ob01", tenant_id:1002, effective_tenant:"ob01", effective_tenant_id:1002, database:"ob_db_mysql_hsd", user:"clhsd@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7fd43819eff0}, exec_ctx.get_errcode()=-4038)
[2023-07-23 11:04:41.128758] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:90) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=24][errcode=-4038] result set open failed, will retry(ret=-4038, cli_ret=-4038, cret=-4038, retry_ctrl_.need_retry()=2)
[2023-07-23 11:04:41.128765] WDIAG [SERVER] response_result (obmp_query.cpp:1292) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6][errcode=-4038] fail response async result(ret=-4038)
[2023-07-23 11:04:41.128769] WDIAG [SERVER] do_process (obmp_query.cpp:846) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038] execute query fail(ret=-4038, timeout_timestamp=1690081573275239)
[2023-07-23 11:04:41.128775] WDIAG [SERVER] do_process (obmp_query.cpp:894) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=4][errcode=-4038]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.128790] TRACE [TRACE] after_process (obmp_base.cpp:156) [80219][T1002_L0_G100][T1002][YB420A01819C-0005FDE730AB57FD-0-0] [lt=6] [packet retry query](TRACE=begin_ts=1690081481127861 2023-07-23 03:04:41.127861|[process_begin] u=0 in_queue_time:7852621, receive_ts:1690081473275239, enqueue_ts:1690081481127849|[start_sql] u=0 addr:{ip:"192.168.1.1", port:42958}|[query_begin] u=1 trace_id:YB420A01819C-0005FDE730AB57FD-0-0|[before_processor_run] u=1 |[session] u=1 sid:3222121549, tenant_id:1002|[pc_choose_plan] u=46 |[check_priv] u=8 |[plan_id] u=1 plan_id:28026|[do_open_plan_begin] u=1 plan_id:28026|[sql_start_stmt_begin] u=1 |[sql_start_stmt_end] u=2 |[exec_plan_begin] u=0 |[exec_plan_end] u=3 |[do_open_plan_end] u=0 |[insert_open] u=1 |[value_start_calc_row] u=3 |[value_after_calc_row] u=8 |[insert_start] u=0 |[get_das_id_begin] u=12 |[get_das_id_end] u=0 |[insert_end] u=3 |[do_local_das_task_begin] u=1 |[do_local_das_task_end] u=237 |[sync_auto_value] u=418 |[close_plan_begin] u=61 |[insert_close] u=1 |[close_das_task_begin] u=0 |[close_das_task_end] u=1 |[start_end_stmt] u=1 |[end_stmt] u=82 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=2 |[query_end] u=32 |[process_end] u=1 run_ts:1690081481127863|total_timeu=929)
[2023-07-23 11:04:41.129170] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307932B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.129691] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307932B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.130190] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307932B-0-0] [lt=1] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.130690] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [3288][TbltLSSrv1][T1002][YB420A01819C-0005FDE71307932B-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.130778] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C8A-0-0] [lt=19][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:41.130969] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:145) [83137][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=18][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT")
[2023-07-23 11:04:41.132927] INFO  [STORAGE] runTimerTask (ob_checkpoint_service.cpp:351) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=6] ====== check clog disk timer task ======
[2023-07-23 11:04:41.132944] INFO  [PALF] get_disk_usage (palf_env_impl.cpp:781) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=13] get_disk_usage(ret=0, capacity(MB):=6758, used(MB):=5348)
[2023-07-23 11:04:41.134115] INFO  [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1297) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] succ to get rec scn(*this={this:0x7fd478e04030, ls_id:{id:1}, tenant_id:1001, state:"F_WORKING", total_tx_ctx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.size():0, max_wait_ckpt_ts_:{val:1690048801926620928}, last_push_gc_task_ts_:1690048803399922, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615}, prev_aggre_rec_scn:{val:18446744073709551615}, uref:3}, aggre_rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.134137] INFO  [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:232) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=22] tx ctx memtable get rec scn(this={ObITable:{this:0x7fd483fe6080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1}, end_scn:{val:1690048802126249}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7fd483fe6080, snapshot_version:{val:1690048802126249}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903})
[2023-07-23 11:04:41.134161] INFO  [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:742) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=20] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903}, flushed_scn_={val:0}, pre_rec_scn_={val:18446744073709551615}, freeze_scn_={val:0}, max_committed_scn_={val:18446744073709551615}, is_frozen_=false, ls_id_={id:1})
[2023-07-23 11:04:41.134178] INFO  [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:441) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7fd4867ec1b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7fd4867f1290, table_type:1, memtable_head:244, memtable_tail:245, t3m:0x7fd4ba7e2030, tables:[null, null, null, null, 0x7fd483fd6080, null, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, tx_data_table:0x7fd4867f2650, ls_tablet_svr:0x7fd4867ec190, slice_allocator:0x7fd4867f2690}, min_rec_scn={val:1690067853855506884}, ls_id_={id:1})
[2023-07-23 11:04:41.134901] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=27] get rec log scn(service_type_=0, rec_log_ts={val:1690081456904556569})
[2023-07-23 11:04:41.134912] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] get rec log scn(service_type_=1, rec_log_ts={val:1690061231333212449})
[2023-07-23 11:04:41.134925] INFO  [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:306) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=12] get rec log scn(service_type_=2, rec_log_ts={val:1690075632203527680})
[2023-07-23 11:04:41.134934] INFO  [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:158) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1690061231333212449}, checkpoint_scn_in_ls_meta={val:1690061231333212449}, ls_id={id:1}, service_type="TRANS_ID")
[2023-07-23 11:04:41.134949] INFO  [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:264) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=11] cannot_recycle_log_size statistics(cannot_recycle_log_size=575073757, threshold=2126008811)
[2023-07-23 11:04:41.134973] INFO  [STORAGE] flush_if_need_ (ob_ls.cpp:1564) [83379][T1001_CKClogDis][T1001][Y0-0000000000000000-0-0] [lt=9] the ls no need flush to advance_checkpoint(get_ls_id()={id:1}, need_flush=false)
[2023-07-23 11:04:41.135038] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2135) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=9] [PALF_DUMP](palf_id=1, self="192.168.1.1:1001", [SlidingWindow]={palf_id:1, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:146980983, max_lsn:99688838460, max_scn:{val:1690081466798649234}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:99688816093}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:99688815606}, last_submit_end_lsn:{lsn:99688816093}, last_submit_log_id:146980951, last_submit_log_pid:1, max_flushed_lsn:{lsn:99688815606}, max_flushed_end_lsn:{lsn:99688816093}, max_flushed_log_pid:1, committed_end_lsn:{lsn:99688816093}, last_slide_log_id:146980951, last_slide_scn:{val:1690081465461187621}, last_slide_lsn:{lsn:99688815606}, last_slide_end_lsn:{lsn:99688816093}, last_slide_log_pid:1, last_slide_log_accum_checksum:2038933450, last_fetch_end_lsn:{lsn:99730759133}, last_fetch_max_log_id:146982999, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd48ddf6530}, [StateMgr]={this:0x7fd48ddf93f0, self:"192.168.1.1:1001", palf_id:1, role:"FOLLOWER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"0.0.0.0:0", leader_epoch:0, is_sync_enabled:true, allow_vote:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:146980952, is_changing_config_with_arb:false, reconfirm_start_time_us:-1, palf_role_change_cb:0x7fd48ddff670, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"192.168.1.1:1001", alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, alive_paxos_replica_num:3, log_ms_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, prev_log_proposal_id:9223372036854775807, applied_alive_paxos_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, applied_alive_paxos_replica_num:3, applied_all_learnerlist:{learner_num:0, learner_array:[]}, prev_lsn:{lsn:18446744073709551615}, prev_mode_pid:9223372036854775807, state:0, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:9223372036854775807, config_seq:-1}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, region:"default_region", paxos_member_region_map:{}, register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7fd48ddf75b0}, [ModeMgr]={palf_id:1, self:"192.168.1.1:1001", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1689887917115365335}, min_block_id:1403, base_lsn_for_block_gc:{lsn:99113742336}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:2, proposal_id:1, prev:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, curr:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:3{server:"10.1.129.154:1001", timestamp:1, flag:0}{server:"10.1.129.155:1001", timestamp:1, flag:0}{server:"192.168.1.1:1001", timestamp:1, flag:0}, log_sync_replica_num:3, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:1, base_lsn:{lsn:99113742336}, prev_log_info:{log_id:-1, lsn:{lsn:18446744073709551615}, scn:{val:18446744073709551615}, log_proposal_id:9223372036854775807, accum_checksum:-1}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1}}}, log_meta_storage:{log_tail:{lsn:1695744}, readable_log_tail:{lsn:1695744}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:99717685248}, min_scn:{val:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1001/1/meta", dir_fd:452, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:65409024, block_header_serialize_buf_:0x7fd48ddfb030}, log_storage:{log_tail:{lsn:99688816093}, readable_log_tail:{lsn:99688816093}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:99650580480}, min_scn:{val:1690080117550018753}, curr_block_id:1485, palf_id:1}, block_mgr:{log_dir:"/home/observer/oceanbase/store/home_ssd/clog/tenant_1001/1/log", dir_fd:471, min_block_id:1403, max_block_id:1486, curr_writable_block_id:1485}, logical_block_size_:67104768, curr_block_writable_size_:28869155, block_header_serialize_buf_:0x7fd48ddfccb0}, palf_epoch:1, this:0x7fd48ddf9af0}, [Reconfirm]={palf_id:1, self:"192.168.1.1:1001", state:"INITED", new_proposal_id:9223372036854775807, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:0, majority_max_log_server:"0.0.0.0:0", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:18446744073709551615}, saved_end_lsn:{lsn:18446744073709551615}, last_submit_prepare_req_time_us:-1, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, has_notify_fetch:false, this:0x7fd48ddf9530})
[2023-07-23 11:04:41.135299] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=248][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:41.135316] INFO  [PALF] check_and_switch_state (palf_handle_impl.cpp:2139) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=25] [PALF_DUMP](palf_id=1, self="192.168.1.1:1001", ack_info_list=[])
[2023-07-23 11:04:41.135331] WDIAG [PALF] get_ack_info_array (log_sliding_window.cpp:2503) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=12][errcode=-4018] match_lsn_map_ get failed(ret=-4018, palf_id=1, self="192.168.1.1:1001", tmp_server={server:"10.1.129.154:1001", timestamp:1, flag:0})
[2023-07-23 11:04:41.135344] INFO  [PALF] report_log_task_trace (log_sliding_window.cpp:3999) [83282][T1001_LogLoop][T1001][Y0-0000000000000000-0-0] [lt=12] current log_task status(palf_id=1, self="192.168.1.1:1001", log_id=146980952, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615}, max_scn:{val:18446744073709551615}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:0, submit_wait_time:0, submit_ts:0, flush_cost:0, flushed_ts:0}, ack_info_list=[])
[2023-07-23 11:04:41.137039] INFO  [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [3291][][T0][YB420A01819C-0005FDE715FDDC48-0-0] [lt=0] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4})
[2023-07-23 11:04:41.137033] INFO  [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C8B-0-0] [lt=19] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1690081481137023})
[2023-07-23 11:04:41.138012] INFO  [SQL.RESV] add_invalid_server_distinctly (ob_sql_context.cpp:196) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=0] add a server to invalid server list(ret=0, invalid_server="10.1.129.154:1001", invalid_servers_=["10.1.129.154:1001"])
[2023-07-23 11:04:41.138050] TRACE [TRACE] after_process (obmp_base.cpp:156) [45604][T1_L0_G100][T1][YB420A01819C-0005FDE72D1C9C87-0-0] [lt=1] [packet retry query](TRACE=begin_ts=1690081481137929 2023-07-23 03:04:41.137929|[process_begin] u=0 in_queue_time:641054, receive_ts:1690081480496875, enqueue_ts:1690081481137919|[start_sql] u=1 addr:{ip:"192.168.1.1", port:43084}|[query_begin] u=0 trace_id:YB420A01819C-0005FDE72D1C9C87-0-0|[before_processor_run] u=2 |[session] u=1 sid:3222085985, tenant_id:1|[calc_partition_location_begin] u=24 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=1 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=1 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=1 |[calc_partition_location_begin] u=0 |[tl_calc_part_id_end] u=0 |[get_location_cache_begin] u=0 |[calc_partition_location_end] u=0 |[get_plan_type_end] u=3 |[pc_choose_plan] u=2 |[check_priv] u=2 |[plan_id] u=0 plan_id:18472|[do_open_plan_begin] u=2 plan_id:18472|[sql_start_stmt_begin] u=0 |[sql_start_stmt_end] u=0 |[exec_plan_begin] u=0 |[parse_job_begin] u=3 |[parse_job_end] u=6 |[remote_task_completed] u=42 ret:-4122, runner_svr:{ip:"10.1.129.154", port:1001}, task:{task_id:{job_id:{execution_id:{addr:{ip:"192.168.1.1", port:1001}, execution_id:18446744073709551615}, job_id:29271}, task_id:0}, runner_svr:{ip:"10.1.129.154", port:1001}, ctrl_svr:{ip:"192.168.1.1", port:1001}}|[exec_plan_end] u=9 |[do_open_plan_end] u=1 |[get_location_cache_begin] u=1 |[close_plan_begin] u=2 |[start_end_stmt] u=0 |[end_stmt] u=0 |[close_plan_end] u=0 |[auto_end_plan_begin] u=0 |[auto_end_plan_end] u=3 |[query_end] u=9 |[process_end] u=1 run_ts:1690081481137932|total_timeu=120)
[2023-07-23 11:04:41.139078] WDIAG [RPC.FRAME] create_request (ob_req_transport.cpp:371) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=25][errcode=-4122]  REACH SYSLOG RATE LIMIT
[2023-07-23 11:04:41.139097] INFO  [PALF] do_fetch_log_ (log_sliding_window.cpp:1927) [83042][T1002_LogLoop][T1002][Y0-0000000000000000-0-0] [lt=0] do_fetch_log_ finished(ret=0, palf_id=1002, self="192.168.1.1:1001", dest="10.1.129.155:1001", fetch_log_count=2048, fetch_start_lsn={lsn:406891815358}, prev_lsn={lsn:406891814580}, fetch_start_log_id=130559082, last_slide_log_id=130559081, fetch_log_size=41943040, accepted_mode_pid=1, last_fetch_req_time=1690081481139095, last_fetch_end_lsn={lsn:406933758398}, last_fetch_max_log_id=130561129, last_fetch_committed_end_lsn={lsn:18446744073709551615}, trigger_type=0, this={palf_id:1002, self:"192.168.1.1:1001", lsn_allocator:{max_log_id:130559082, max_lsn:406891815781, max_scn:{val:1690081465085443984}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:406891815781}, reserved_buffer_size:41943040, available_buffer_size:41943040}, last_submit_lsn:{lsn:406891815358}, last_submit_end_lsn:{lsn:406891815781}, last_submit_log_id:130559082, last_submit_log_pid:1, max_flushed_lsn:{lsn:406891815358}, max_flushed_end_lsn:{lsn:406891815781}, max_flushed_log_pid:1, committed_end_lsn:{lsn:406891815358}, last_slide_log_id:130559081, last_slide_scn:{val:1690081465077463222}, last_slide_lsn:{lsn:406891814580}, last_slide_end_lsn:{lsn:406891815358}, last_slide_log_pid:1, last_slide_log_accum_checksum:2796067235, last_fetch_end_lsn:{lsn:406933758398}, last_fetch_max_log_id:130561129, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, this:0x7fd45c242530})
[2023-07-23 11:04:41.147415] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1574) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] gts not ready(ret=-4283, retry_times=102)
[2023-07-23 11:04:41.147426] WDIAG [STORAGE.TRANS] acquire_global_snapshot__ (ob_trans_service_v4.cpp:1596) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=11][errcode=-4283] acquire global snapshot fail(ret=-4283, gts_ahead=0, expire_ts=1690081500606537, now=[mts=1690081481091427], now0=[mts=1690081481091427], snapshot={val:18446744073709551615}, uncertain_bound=0)
[2023-07-23 11:04:41.147439] WDIAG [STORAGE.TRANS] get_read_snapshot (ob_tx_api.cpp:592) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=12][errcode=-4283] acquire global snapshot fail(ret=-4283, tx={this:0x7fd4f12b38a0, tx_id:{txid:44908689}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481091009, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.147471] WDIAG [SQL.EXE] stmt_setup_snapshot_ (ob_sql_trans_control.cpp:679) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=29][errcode=-4283] fail to get snapshot(ret=-4283, local_ls_id={id:-1}, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0})
[2023-07-23 11:04:41.147487] WDIAG [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:531) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=16][errcode=-4283] fail to exec stmt_setup_snapshot_(session, das_ctx, plan, plan_ctx, txs)(ret=-4283, session_id=1, *tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908689}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481091009, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2})
[2023-07-23 11:04:41.147512] INFO  [SQL.EXE] start_stmt (ob_sql_trans_control.cpp:589) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=24] start stmt(ret=-4283, auto_commit=true, session_id=1, snapshot={this:0x7fd5b2852cc8, valid:false, source:0, core:{version:{val:18446744073709551615}, tx_id:{txid:0}, scn:-1}, uncertain_bound:0, snapshot_lsid:{id:-1}, parts:[]}, savepoint=1690081481051102, tx_desc={this:0x7fd4f12b38a0, tx_id:{txid:44908689}, state:1, addr:"192.168.1.1:1001", tenant_id:1, session_id:1, assoc_session_id:1, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615}, snapshot_scn:0, active_scn:-1, op_sn:2, alloc_ts:1690081481091009, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615}, commit_cb:null, cluster_id:3, cluster_version:17179934720, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, plan_type=4, stmt_type=2, has_for_update=false, query_start_time=1690081470676545, use_das=false, nested_level=0, session={this:0x7fd521d0a0d8, id:1, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7fd4f12b38a0}, plan=0x7fd507268050, consistency_level_in_plan_ctx=3, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]})
[2023-07-23 11:04:41.147553] WDIAG [SQL] start_stmt (ob_result_set.cpp:317) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=41][errcode=-4283] fail to start stmt(ret=-4283, phy_plan->get_dependency_table()=[{table_id:309, schema_version:1686281721492736, object_type:1, is_db_explicit:false, is_existed:true}])
[2023-07-23 11:04:41.147564] WDIAG [SQL] do_open_plan (ob_result_set.cpp:496) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10][errcode=-4283] fail start stmt(ret=-4283)
[2023-07-23 11:04:41.147572] WDIAG [SQL] open (ob_result_set.cpp:157) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] execute plan failed(ret=-4283)
[2023-07-23 11:04:41.147579] WDIAG [SERVER] open (ob_inner_sql_result.cpp:153) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=7][errcode=-4283] open result set failed(ret=-4283)
[2023-07-23 11:04:41.147585] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:648) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=4][errcode=-4283] result set open failed(ret=-4283, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"})
[2023-07-23 11:04:41.147594] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:783) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=8][errcode=-4283] execute failed(ret=-4283, tenant_id=1, executor={ObIExecutor:, sql:"INSERT INTO __all_space_usage (tenant_id, svr_ip, svr_port, start_seq, file_type, data_size, used_size) VALUES (1, '192.168.1.1', 2882, 3, 'tenant file data', 260046848, 260046848) ON DUPLICATE KEY UPDATE data_size = 260046848, used_size = 260046848"}, retry_cnt=99, local_sys_schema_version=1686562084609584, local_tenant_schema_version=1686562084609584)
[2023-07-23 11:04:41.147604] INFO  [SERVER] sleep_before_local_retry (ob_query_retry_ctrl.cpp:91) [3294][DiskUseReport][T1][YB420A01819C-0005FDE72A9B82D4-0-0] [lt=10] will sleep(sleep_us=99000, remain_us=19528933, base_sleep_us=1000, retry_sleep_type=1, v.stmt_retry_times_=99, timeout_timestamp=1690081500676537)
[2023-07-23 11:04:41.149607] INFO  [SERVER] process (obmp_quit.h:55) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E361-0-0] [lt=10] quit
[2023-07-23 11:04:41.149621] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:152) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E361-0-0] [lt=12][errcode=-4009] sock has error(ret=-4009)
[2023-07-23 11:04:41.149602] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:818) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4015] sock error detect by epoll(mask=8197, *s={this:0x7fd39cc7c170, fd:392, err:5, last_decode_time_:1690081481149582, last_write_time_:1690081481108138, read_buffer_.get_consume_sz():235, get_pending_flag():1, get_trace_id():YB420A01819C-0005FDE72D09E361-0-0})
[2023-07-23 11:04:41.149626] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:929) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E361-0-0] [lt=6][errcode=-4009] write response fail(ret=-4009)
[2023-07-23 11:04:41.149632] WDIAG [SERVER] response (obmp_base.cpp:86) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E361-0-0] [lt=5][errcode=-4009] failed to flush_buffer(ret=-4009)
[2023-07-23 11:04:41.149636] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:48) [3768][T1_L0_G0][T1][YB420A01819C-0005FDE72D09E361-0-0] [lt=3][errcode=-4009] response rpc result fail(ret=-4009)
[2023-07-23 11:04:41.149643] INFO  [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:231) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=33] kill and revert session(conn.sessid_=3222110435, proxy_sessid=0, server_id=3, ret=0)
[2023-07-23 11:04:41.149662] INFO  [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:791) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=18] can close safely, do destroy(*s={this:0x7fd39cc7c170, fd:392, err:5, last_decode_time_:1690081481149582, last_write_time_:1690081481108138, read_buffer_.get_consume_sz():240, get_pending_flag():1, get_trace_id():Y0-0000000000000000-0-0})
[2023-07-23 11:04:41.149692] INFO  [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:144) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=19] unlock session of tenant(conn.sessid_=3222110435, proxy_sessid=0, conn.tenant_id_=1)
[2023-07-23 11:04:41.149708] INFO  [RPC.OBMYSQL] sm_conn_log_close (obsm_conn_callback.cpp:159) [3384][sql_nio6][T0][Y0-0000000000000000-0-0] [lt=14] connection close(sessid=3222110435, proxy_sessid=0, tenant_id=1, server_id=3, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, ret=0)
[2023-07-23 11:04:41.149732] INFO  [SERVER] run (obmp_disconnect.cpp:78) [3768][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=5] free session successfully(sessid=3222110435)
[2023-07-23 11:04:41.151326] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:102) [3767][T1_L0_G0][T1][YB420A01819C-0005FDE72D1C9C8B-0-0] [lt=18][errcode=-4008] execute fail(ret=-4008)
[2023-07-23 11:04:41.151710] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1, gts_local_cache={srr:[mts=1690081465129733], gts:1690081465129634252, latest_srr:[mts=1690081481151700]})
[2023-07-23 11:04:41.151727] INFO  [STORAGE.TRANS] refresh_gts_location_ (ob_gts_source.cpp:580) [3319][TsMgr][T0][Y0-0000000000000000-0-0] [lt=0] gts nonblock renew success(ret=0, tenant_id=1001, gts_local_cache={srr:[mts=1690081465067099], gts:1690081465066867154, latest_srr:[mts=1690081481151722]})
[2023-07-23 11:04:41.151734] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:83) [3272][SysLocAsyncUp0][T0][YB420A01819C-0005FDE62357B06C-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)