observer.log.20240819203122597:[2024-08-19 20:31:14.982282] INFO [SQL.JO] compute_table_location (ob_join_order.cpp:277) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=1] succeed to calculate base table sharding info(table_id=18446744073709551611, ref_table_id=776103, is_global_index=false) observer.log.20240819203122597:[2024-08-19 20:31:14.982304] INFO [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:12120) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=14] total rowcount, use statistics(table_meta_info_.table_row_count_=170311, table_meta_info_.average_row_size_=5.240000000000000000e+02, table_meta_info_.micro_block_count_=212, table_meta_info_.part_size_=8.924296400000000000e+07) observer.log.20240819203122597:[2024-08-19 20:31:14.982557] INFO [SQL.JO] compute_table_location (ob_join_order.cpp:277) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=8] succeed to calculate base table sharding info(table_id=18446744073709551610, ref_table_id=776200, is_global_index=false) observer.log.20240819203122597:[2024-08-19 20:31:14.982566] INFO [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:12120) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=5] total rowcount, use statistics(table_meta_info_.table_row_count_=448, table_meta_info_.average_row_size_=3.860000000000000000e+02, table_meta_info_.micro_block_count_=2, table_meta_info_.part_size_=1.729280000000000000e+05) observer.log.20240819203122597:[2024-08-19 20:31:14.982706] INFO [SQL.JO] compute_table_location (ob_join_order.cpp:277) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=5] succeed to calculate base table sharding info(table_id=18446744073709551609, ref_table_id=776308, is_global_index=false) observer.log.20240819203122597:[2024-08-19 20:31:14.982712] INFO [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:12120) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=3] total rowcount, use statistics(table_meta_info_.table_row_count_=4445, table_meta_info_.average_row_size_=1.350000000000000000e+02, table_meta_info_.micro_block_count_=6, table_meta_info_.part_size_=6.000750000000000000e+05) observer.log.20240819203122597:[2024-08-19 20:31:14.982837] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:193) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=3] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:290016}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1721789058591755339, delete_commit_version:-1}, snapshot_version=4611686018427387895) observer.log.20240819203122597:[2024-08-19 20:31:14.982856] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:193) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=8] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:1152921504607025885}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1721789058591755339, delete_commit_version:-1}, snapshot_version=4611686018427387895) observer.log.20240819203122597:[2024-08-19 20:31:14.989639] INFO [STORAGE] check_status_for_new_mds (ob_tablet_create_delete_helper.cpp:193) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=4] refresh tablet status cache(ret=0, ls_id={id:1001}, tablet_id={id:289971}, tablet_status_cache={tablet_status:{val:1, str:"NORMAL"}, create_commit_version:1721789047773489198, delete_commit_version:-1}, snapshot_version=4611686018427387895) observer.log.20240819203122597:[2024-08-19 20:31:14.989899] INFO [SQL.JO] compute_table_location (ob_join_order.cpp:277) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=10] succeed to calculate base table sharding info(table_id=18446744073709551609, ref_table_id=776308, is_global_index=false) observer.log.20240819203122597:[2024-08-19 20:31:14.989911] INFO [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:12120) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=8] total rowcount, use statistics(table_meta_info_.table_row_count_=4445, table_meta_info_.average_row_size_=1.350000000000000000e+02, table_meta_info_.micro_block_count_=6, table_meta_info_.part_size_=6.000750000000000000e+05) observer.log.20240819203122597:[2024-08-19 20:31:15.000429] WDIAG update_qc_error_code (ob_px_util.h:591) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=8][errcode=-4016] observer.log.20240819203122597:[2024-08-19 20:31:15.000464] WDIAG [SQL.ENG] process_sqc_finish_msg_once (ob_px_scheduler.cpp:410) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=34][errcode=-4016] sqc fail, abort qc(pkt={dfo_id:1, sqc_id:0, rc:-4016, das_retry_rc:0, sqc_affected_rows:0}, ret=-4016, sqc_addr="172.31.44.197:2882") observer.log.20240819203122597:[2024-08-19 20:31:15.000474] WDIAG [SQL.ENG] on_sqc_finish_msg (ob_px_scheduler.cpp:296) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=9][errcode=-4016] failed to process_sqc_finish_msg_once observer.log.20240819203122597:[2024-08-19 20:31:15.000479] WDIAG [SQL.DTL] process (ob_dtl_channel_loop.cpp:209) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=4][errcode=-4016] process message in channel fail(header={hlen:16, type:12, nbody:411, checksum:0}, ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000486] WDIAG [SQL.DTL] process_base (ob_dtl_channel_loop.cpp:272) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=6][errcode=-4016] fail process channel(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000489] WDIAG [SQL.ENG] fetch_rows (ob_px_fifo_coord_op.cpp:212) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=3][errcode=-4016] fail process message(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000493] WDIAG [SQL.ENG] fetch_rows (ob_px_fifo_coord_op.cpp:244) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=3][errcode=-4016] QC get error code(ret=-4016, ret_terminate=0) observer.log.20240819203122597:[2024-08-19 20:31:15.000498] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1194) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=4][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f261dd0d970}, id=0, op_name="PHY_PX_FIFO_COORD") observer.log.20240819203122597:[2024-08-19 20:31:15.000505] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1499) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=7][errcode=-4016] get next batch failed(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000508] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:411) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=3][errcode=-4016] get next row from exec result failed(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000512] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1448) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=3][errcode=-4016] do dml query failed(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000515] WDIAG [SQL] open_result (ob_result_set.cpp:214) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=2][errcode=-4016] fail to drive dml query(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000518] WDIAG [SQL] open (ob_result_set.cpp:160) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=2][errcode=-4016] open result set failed(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000536] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:63) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=17][errcode=-4016] failed to do result set open(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000541] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:947) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=4][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false, THIS_WORKER.can_retry()=true, v.ctx_.multi_stmt_item_={is_part_of_multi_stmt:false, seq_num:0, sql:"INSERT INTO gs_merchant_msg ( msg_id, merchant_id, emp_id, add_time, read_time, state ) observer.log.20240819203122597:[2024-08-19 20:31:15.000581] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:646) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=11] try unregister deadlock detecotr cause conflict array is empty(step=9, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f27ae69e268, id:3224524003, deser:false, tenant:"game", tenant_id:1002, effective_tenant:"game", effective_tenant_id:1002, database:"base_game", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f283a19a520}, desc={this:0x7f283a19a520, tx_id:{txid:1361670333}, state:1, addr:"172.31.32.214:2882", tenant_id:1002, session_id:3224524003, assoc_session_id:3224524003, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1724070662879439, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1719287282, cluster_version:17180000519, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[]) observer.log.20240819203122597:[2024-08-19 20:31:15.000648] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:670) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=66] maintain deadlock info(step=9, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f27ae69e268, id:3224524003, deser:false, tenant:"game", tenant_id:1002, effective_tenant:"game", effective_tenant_id:1002, database:"base_game", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f283a19a520}, desc={this:0x7f283a19a520, tx_id:{txid:1361670333}, state:1, addr:"172.31.32.214:2882", tenant_id:1002, session_id:3224524003, assoc_session_id:3224524003, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:1724070662879439, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1719287282, cluster_version:17180000519, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, is_rollback=true, conflict_txs=[]) observer.log.20240819203122597:[2024-08-19 20:31:15.000662] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1129) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=12] end stmt(ret=0, plain_select=false, stmt_type=2, savepoint=1724070674995769, tx_desc={this:0x7f283a19a520, tx_id:{txid:1361670333}, state:1, addr:"172.31.32.214:2882", tenant_id:1002, session_id:3224524003, assoc_session_id:3224524003, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:3, alloc_ts:1724070662879439, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:86400000000, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1719287282, cluster_version:17180000519, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, cflict_txs:[], abort_cause:0, commit_expire_ts:-1, commit_task_.is_registered():false, ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], cflict_txs:[]}, rollback=true, session={this:0x7f27ae69e268, id:3224524003, deser:false, tenant:"game", tenant_id:1002, effective_tenant:"game", effective_tenant_id:1002, database:"base_game", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f283a19a520}, exec_ctx.get_errcode()=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000729] WDIAG [SERVER] response_result (ob_async_plan_driver.cpp:92) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=17][errcode=-4016] result set open failed, let's leave process(). EndTransCb will clean this mess(ret=-4016, cli_ret=-4016, cret=-4016, retry_ctrl_.need_retry()=0) observer.log.20240819203122597:[2024-08-19 20:31:15.000778] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:319) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=5] sending error packet(ob_error=-4016, client error=-4016, extra_err_info=NULL, lbt()="0x1203501c 0x9db8660 0x9d65e29 0x9d61bdf 0x4c3ce64 0x4c36a3c 0x4c2e2b8 0x4c296a0 0x9a8d0d4 0x123077c3 0x7f2b65f6844b 0x7f2b65ca352f") observer.log.20240819203122597:[2024-08-19 20:31:15.000807] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:502) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=6] dump txn free route audit_record(value=1048581, session->get_sessid()=3224524003, session->get_proxy_sessid()=12402668002485684151) observer.log.20240819203122597:[2024-08-19 20:31:15.000813] WDIAG [SERVER] response_result (obmp_query.cpp:1273) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=5][errcode=-4016] fail response async result(ret=-4016) observer.log.20240819203122597:[2024-08-19 20:31:15.000852] WDIAG [SERVER] do_process (obmp_query.cpp:808) [10300][T1002_L0_G0][T1002][YB42AC1F20D6-00061D78E87FB516-0-0] [lt=38][errcode=-4016] execute query fail(ret=-4016, timeout_timestamp=1724070734979203)